Run 3d games remotely (even behind a nat) with sound (Onlive DiY)

Question

D you know Onlive? Here is a DIY howto, this is what i'm using to play games remotely on a poor netbook
Required packages:
Server: virtualgl,ffmpeg,socat,sshd
Client: virtualgl,mplayer,ssh,socat
Server: in /etc/ssh/sshd_config:
"AcceptEnv ALSA_CARD"
The user who wants to play remotely has to execute the following script on (say) the poor netbook:
(configure $user,$server and $sshport first), and as soon as it gains access to the remote shell, it has to execute:
vglrun (parameters) /path/to/3dgame
Main Script
#!/bin/bash
user=remote_username
server=remote_server
sshport=22
ServerSetup="\
killall socat ffmpeg ; sleep 1 ; killall -9 socat ffmpeg; \
echo inserting loopback module, ask for sudo pass:; \
sudo modprobe snd-aloop; \
socat UDP4-LISTEN:6000,fork,reuseaddr TCP:127.0.0.1:5000 &>/dev/null & \
ffmpeg -f alsa -ac 2 -i hw:Loopback,1,0 -acodec libmp3lame -b 128k -f rtp rtp://127.0.0.1:6000 &>/dev/null\
echo "Executing ServerSetup on " $Server:$sshport ...
ssh -f $user@$server -oPort=$sshport "$(echo $ServerSetup)"
killall mplayer socat ; sleep 1 ; killall -9 mplayer socat
echo "o=- 0 0 IN IP4 127.0.0.1" >/tmp/stream.sdp
echo "c=IN IP4 127.0.0.1" >>/tmp/stream.sdp
echo "m=audio 6000 RTP/AVP 14" >>/tmp/stream.sdp
socat TCP4-LISTEN:5000,fork,reuseaddr UDP4:localhost:6000 &>/dev/null &
sh -c "sleep 20 ; mplayer /tmp/stream.sdp -really-quiet </dev/null" &
ALSA_CARD=Loopback vglconnect -s $user@$server -o SendEnv=ALSA_CARD -p $sshport -R localhost:5000:localhost:5000
Explaination
Graphic streaming is very easy and totally managed by virtualgl as video data is taken from the server by using a seamless VirtualGL window,
vglconnect (part of virtualgl package) is an ssh wrapper that set-up port forwarding to tunnel frames,
no problem here for nat environments.
When comes to audio, the 3d application on the server will output all audio data to a virtual loopback device,
this is done by modprobing snd-aloop and then set the environment variable ALSA_CARD to "Loopback"
Sound is then encoded by ffmpeg into an mp3 streamed via rtp udp transport for lower delay.
Finally, that audio will be played back on the netbook via a background mplayer.
It will take as argument a text file (compiled by a bunch of echo runtime) describing the stream.
(ffplay would do it as well, but how to disable the spectrum visualization?)
Making audio work for client behind a nat is somehow tricky, because ssh only forwards TCP connections and our stream works over UDP,
but with socat we will 'convert' UDP to TCP (ssh here!) to UDP again so that we can use ssh tunnelling facilities while in the TCP domain:
server: ffmpeg will stream to localhost, udp port 6000
server: socat will forward from localhost, udp port 6000 to localhost tcp port 5000
server: vglconnect (ssh wrapper) will forward all tcp traffic directed to localhost:5000(tcp) to the remote endpoint (client), port 5000(tcp)
client: socat will forward from localhost, tcp port 5000 to localhost udp port 6000
client: finally mplayer will play from localhost port 6000
Note that mplayer will run on the client, but the last step we'll do in the script is to open an ssh connection into the server,
and that connection is necessary for mplayer to work as we'll setup port redirection there.
So, to keep things easy from the user point of view, mplayer is started early in background with a delay (20secs)
(read: your audio will start after a while)
If you are not behind a nat or prefer to use a vpn, you can (but it is not necessary at all)
* comment all of the socat commands,
* point ffmpeg streaming to the client ip,
* change the line that contains "c=IN IP4 127.0.0.1" to the client ip too.
Drawbacks of the script:
.It will kill any socat,ffmpeg and mplayer instances at startup (any idea?)
.No process has to listen on port 5000 and 6000 (tcp,udp), change the script or use some vars if you need it.
.Some programs (eg: braid) don't like alsa loopback device and outputs some garbage sound
.ALSA_CARD trick isn't going to work if you forced a !default output device in your .asoundrc (server side)
What is missing:
.A way to terminate leftover processes when we're done with playing
Highly suggested:
.remove "&>/dev/null" occurrences from the script if you run into problems
.Setup ssh key authentication
.Avoid sound skip by running ffmpeg with realtime priority (schedtool -n -19 -F -p 10 -e ffmpeg...)
Side notes:
.This thing doesn't work very well with virtualized environments, virtualbox client on the same server machine performs poorly, think that a real n280 netbook over a real 10Mbps link is smoother.
.Audio latency is about 300ms (i know it is not THAT low, but acceptable)
example code for a 10Mbit network which take about 600KBps at 1024x600
vglrun -np 2 -c jpeg -q 40 -samp 1 -fps 20 ./aquaria
-np 2 = use 2 threads to encode frames
-q 40 = jpeg quality=40
-samp 1 = Chrominance subsampling factor
-fps 20 = limit the framerate to 20fps
Last edited by kokoko3k (2012-01-24 14:11:44)

Answer 1

Hi kokoko3k thanks for the script. I had some troubles to start with, but hacked around and got it going.
I am using AMD64 Ubuntu based distributions for both client and server.
Anyways I thought I would post the (slightly) modified script in case it helps others out there.
#!/bin/bash
#Prereqs:
#Server: virtualgl,ffmpeg,socat,sshd,libavcodec-extra-53
#Client: virtualgl,mplayer,ssh,socat
#Server: in /etc/ssh/sshd_config:
#"AcceptEnv ALSA_CARD"
#Server: add to /etc/sudoers file:
#<user> ALL=(ALL) NOPASSWD: /sbin/modprobe
user=<user>
server=<Server Hostname>
sshport=22
ServerSetup="\
killall socat ffmpeg ; sleep 1 ; killall -9 socat ffmpeg; \
sudo modprobe snd-aloop; \
socat UDP4-LISTEN:6000,fork,reuseaddr TCP:127.0.0.1:5000 &>/dev/null & \
ffmpeg -f alsa -ac 2 -i hw:Loopback,1,0 -acodec libmp3lame -b 128k -f rtp rtp://127.0.0.1:6000 &>/dev/null\
echo "Executing ServerSetup on " $server:$sshport ...
ssh -f $user@$server -oPort=$sshport "$(echo $ServerSetup)"
killall vglclient mplayer socat ; sleep 1 ; killall -9 vglclient mplayer socat
echo "o=- 0 0 IN IP4 127.0.0.1" >/tmp/stream.sdp
echo "c=IN IP4 127.0.0.1" >>/tmp/stream.sdp
echo "m=audio 6000 RTP/AVP 14" >>/tmp/stream.sdp
socat TCP4-LISTEN:5000,fork,reuseaddr UDP4:localhost:6000 &>/dev/null &
sh -c "sleep 60 ; mplayer /tmp/stream.sdp -really-quiet </dev/null" &
ALSA_CARD=Loopback /opt/VirtualGL/bin/vglconnect -x -s $user@$server -o SendEnv=ALSA_CARD -p $sshport -R localhost:5000:localhost:5000
Last edited by Ken (2012-02-27 18:18:33)