Game recording issues with ffmpeg

Question

Hello, I'm trying to create a script to record windowed games with ffmpeg.
This is what I have so far:
#!/bin/sh
INFO=$(xwininfo -frame)
WIN_GEO=$(echo $INFO | grep -oEe 'geometry [0-9]+x[0-9]+' | grep -oEe '[0-9]+x[0-9]+')
WIN_XY=$(echo $INFO | grep -oEe 'Corners:\s+\+[0-9]+\+[0-9]+' | grep -oEe '[0-9]+\+[0-9]+' | sed -e 's/+/,/' )
ffmpeg -f x11grab -y -r 60 -s $WIN_GEO -i :0.0+$WIN_XY -f alsa -i pulse -crf 18 -preset ultrafast -threads 8 -async 5000 $1.mkv
It allows me to click on a window and record it - it works, but has some issues:
> I manually have to redirect the game audio with pavucontrol to be able to record it
> Audio gets out of sync, I tried many options with no luck
> I can't find a way to record both game audio and microphone audio
So,
> Could the script be improved somehow for better quality/performance?
> How can I prevent the audio going out of sync?
> How can I record both game video, game audio and microphone audio at the same time?
Thanks.

Answer 1

SuperV1234 wrote: > I manually have to redirect the game audio with pavucontrol to be able to record it
Usually, PulseAudio will remember the last used device. Once you run ffmpeg once, PA should remember that it listened on the monitor input and use that.
SuperV1234 wrote: > Audio gets out of sync, I tried many options with no luck
I got this problem too a while back. Try removing the -async option, it's deprecated and caused more trouble than I had. Also, make sure you always have enough processing power to process everything in time, otherwise it will become out of sync quickly. ffmpeg isn't smart enough yet to fill the gaps, so if you can't grab the audio fast enough (pulseaudio glitch), can't encode the video fast enough 100% of the time, or anything else, it will lose frames and result in a delay. Even funnier, also make sure the drive you output to is fast enough to save the file so ffmpeg never block on writing the file to disk (I used to send my encode directly to a streaming server, and the network latency messed it up).
SuperV1234 wrote: > I can't find a way to record both game audio and microphone audio
You can set this up easily with either PA or Jack. Jack is a bit easier to use, but here's the way I do it myself:
- Create a dummy output device (module-null-sink)
- Use a loopback to play your microphone in the dummy output (module-loopback)
- Use a loopback to play your sound card monitor into the dummy output (module-loopback again)
- Make ffmpeg record from the monitor of the dummy output.
You can also juste encode 2 audio streams using ffmpeg (-f alsa -i pulse twice, or -f pulse -i "your pulseaudio input") and edit it after.
I think jack would use less CPU to do the same thing (audio streams are piped and mixed directly in the right place instead of using loopbacks and dummy outputs)
SuperV1234 wrote: > Could the script be improved somehow for better quality/performance?
- Unless you really want 60 FPS output, I'd suggest reducing the FPS to 30. Keep the 60 FPS input, but output into 30 FPS. It will use half the processing power and there will be almost no noticable difference.
- If you really want 60 FPS output and know how it will be displayed, you can actually encode in interlaced video (60i), which looks like 60 FPS but weights about the same as 30 FPS.
- You can increase the quality, using something like -preset fast or even -preset medium given you have enough spare processing. Remember, encoding speed should be constant. If you record 30 FPS, you have to output 30 FPS all the time!
- Like said by skottish, try not using CFR. It will cause the encoder to have to spend more time one some frames and this extra processing might be enough to cause FPS drop.
- Tune your x264 options (-x264args). Options reference
- Try reducing the number of threads: make room for your game and give a ffmpeg as few cores as possible. using 8 cores will make it have full processing power, but there are some cases where your game will have CPU spikes and ffmpeg will wait for a single thread to finish a frame and get out of sync. They will be rescheduled to another core, but this has some overhead, and x264 encoding uses the CPU cache a lot with inline assembler code. Having as few context switch as possible for ffmpeg is preferable over having two ffmpeg threads encoding on the same CPU.
I hope this helps you a bit. Unfortunately, ffmpeg is still pretty bad at recording because of audio/video sync issues In the end, there will always be sync issues after a while, so I suggest you restart ffmpeg once in a while. I opened a bug on their tracker a while ago about similar issues. Never got any attention, no comment, not even moved from new to open. They need to fix the input buffer size in a way they always grab exactly N frames of video and audio. They seem to use some timecodes but it looks like the muxer often fails to resync them back. I'd fix it myself, but ffmpeg is way too big for me to find what I need, especially for a bug that seem to have existed forever and still not being fixed or even addressed.
Last edited by Max-P (2013-07-22 19:16:12)