Recent

Author Topic: Using ffmpeg to capture video and audio on Ubuntu 20, how to set arguments  (Read 5116 times)

BosseB

  • Sr. Member
  • ****
  • Posts: 468
I am making a scheduler in Lazarus for capturing streaming video nightly at specific times and durations.
The scheduler needs to run an ffmpeg command via TProcess for catching the video from a specific screen region (size about 860x480 pixels) plus the audio from the browser playing the stream.
I have found the settings needed for ffmpeg so it works on Windows but the target is to run on Linux (Ubuntu 20).

This is what I use on Windows and want to replicate on Linux:
Code: Text  [Select][+][-]
  1. ffmpeg -hide_banner -f gdigrab -framerate 25 -offset_x 564 -offset_y 85 -video_size 854x480 -i desktop  -f dshow -i audio="virtual-audio-capturer" -vcodec libx264 -pix_fmt yuv420p -t 60 output10.mp4

For Linux I have not (yet) found any working command line for ffmpeg, either the command is rejected because of some argument error or else it does run but the resulting video only contains audio with a black video...

I would appreciate a suggestion for a ffmpeg command that can extract both video and audio with the following constraints:
- the captured region shall be set by offset x and y and size widthx heighty
- the audio capture shall be set to be whatever is sent to the speaker, not picked up by the microphone...

In order to check available devices I have run this command on Ubuntu 20:
Code: Text  [Select][+][-]
  1. $ ffmpeg -hide_banner -devices
  2. Devices:
  3.  D. = Demuxing supported
  4.  .E = Muxing supported
  5.  --
  6.  DE alsa            ALSA audio output
  7.   E caca            caca (color ASCII art) output device
  8.  DE fbdev           Linux framebuffer
  9.  D  iec61883        libiec61883 (new DV1394) A/V input device
  10.  D  jack            JACK Audio Connection Kit
  11.  D  kmsgrab         KMS screen capture
  12.  D  lavfi           Libavfilter virtual input device
  13.  D  libcdio          
  14.  D  libdc1394       dc1394 v.2 A/V grab
  15.  D  openal          OpenAL audio capture device
  16.   E opengl          OpenGL output
  17.  DE oss             OSS (Open Sound System) playback
  18.  DE pulse           Pulse audio output
  19.   E sdl,sdl2        SDL2 output device
  20.  DE sndio           sndio audio playback
  21.  DE video4linux2,v4l2 Video4Linux2 output device
  22.  D  x11grab         X11 screen capture, using XCB
  23.   E xv              XV (XVideo) output device
  24.  
So I use x11grab for video and alsa/pulse for audio. (Don't really know what all these mean.)

I have found one command that does not result in errors but produces a video containing a black screen and audio from the stream but mixed with surrounded audio from the built-in micophone. Obviously I only want the stream audio.
Here an example run:
Code: Text  [Select][+][-]
  1. $ ffmpeg -hide_banner -f x11grab -framerate 25 -video_size 622x484 -i :0+1100,400+nomouse -f alsa -i pulse  -pix_fmt yuv420p -c:v libx264 -c:a libmp3lame  -q:v 1 -t 20 testvideo11.mp4
  2. Input #0, x11grab, from ':0+1100,400+nomouse':
  3.   Duration: N/A, start: 1614180307.110345, bitrate: N/A
  4.     Stream #0:0: Video: rawvideo (BGR[0] / 0x524742), bgr0, 622x484, 25 fps, 25 tbr, 1000k tbn, 1000k tbc
  5. Guessed Channel Layout for Input Stream #1.0 : stereo
  6. Input #1, alsa, from 'pulse':
  7.   Duration: N/A, start: 1614180307.299369, bitrate: 1536 kb/s
  8.     Stream #1:0: Audio: pcm_s16le, 48000 Hz, stereo, s16, 1536 kb/s
  9. Stream mapping:
  10.   Stream #0:0 -> #0:0 (rawvideo (native) -> h264 (libx264))
  11.   Stream #1:0 -> #0:1 (pcm_s16le (native) -> mp3 (libmp3lame))
  12. Press [q] to stop, [?] for help
  13. [swscaler @ 0x556669c07fc0] Warning: data is not aligned! This can lead to a speed loss
  14. [libx264 @ 0x556669bf9d40] -qscale is ignored, -crf is recommended.
  15. [libx264 @ 0x556669bf9d40] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX
  16. [libx264 @ 0x556669bf9d40] profile High, level 3.0
  17. [libx264 @ 0x556669bf9d40] 264 - core 155 r2917 0a84d98 - H.264/MPEG-4 AVC codec - Copyleft 2003-2018 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=12 lookahead_threads=2 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
  18. Output #0, mp4, to 'testvideo11.mp4':
  19.   Metadata:
  20.     encoder         : Lavf58.29.100
  21.     Stream #0:0: Video: h264 (libx264) (avc1 / 0x31637661), yuv420p(progressive), 622x484, q=-1--1, 25 fps, 12800 tbn, 25 tbc
  22.     Metadata:
  23.       encoder         : Lavc58.54.100 libx264
  24.     Side data:
  25.       cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: -1
  26.     Stream #0:1: Audio: mp3 (libmp3lame) (mp4a / 0x6134706D), 48000 Hz, stereo, s16p
  27.     Metadata:
  28.       encoder         : Lavc58.54.100 libmp3lame
  29. [alsa @ 0x556669bd3840] Thread message queue blocking; consider raising the thread_queue_size option (current value: 8)
  30. frame=  500 fps= 25 q=-1.0 Lsize=     454kB time=00:00:20.01 bitrate= 186.0kbits/s speed=0.995x    
  31. video:121kB audio:313kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 4.688069%
  32. [libx264 @ 0x556669bf9d40] frame I:2     Avg QP:18.39  size: 31961
  33. [libx264 @ 0x556669bf9d40] frame P:134   Avg QP:23.16  size:   305
  34. [libx264 @ 0x556669bf9d40] frame B:364   Avg QP:27.21  size:    51
  35. [libx264 @ 0x556669bf9d40] consecutive B-frames:  1.4%  3.6%  3.0% 92.0%
  36. [libx264 @ 0x556669bf9d40] mb I  I16..4: 25.7% 38.2% 36.1%
  37. [libx264 @ 0x556669bf9d40] mb P  I16..4:  0.3%  0.2%  0.1%  P16..4:  2.3%  0.6%  0.4%  0.0%  0.0%    skip:96.2%
  38. [libx264 @ 0x556669bf9d40] mb B  I16..4:  0.1%  0.0%  0.0%  B16..8:  1.2%  0.1%  0.0%  direct: 0.1%  skip:98.5%  L0:30.0% L1:67.5% BI: 2.5%
  39. [libx264 @ 0x556669bf9d40] 8x8 transform intra:34.6% inter:68.8%
  40. [libx264 @ 0x556669bf9d40] coded y,uvDC,uvAC intra: 32.3% 27.5% 18.3% inter: 0.3% 0.4% 0.0%
  41. [libx264 @ 0x556669bf9d40] i16 v,h,dc,p: 54% 42%  1%  3%
  42. [libx264 @ 0x556669bf9d40] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 30% 43% 18%  2%  1%  2%  2%  1%  2%
  43. [libx264 @ 0x556669bf9d40] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 35% 34% 12%  2%  3%  4%  3%  4%  3%
  44. [libx264 @ 0x556669bf9d40] i8c dc,h,v,p: 67% 19% 13%  1%
  45. [libx264 @ 0x556669bf9d40] Weighted P-Frames: Y:0.0% UV:0.0%
  46. [libx264 @ 0x556669bf9d40] ref P L0: 77.0%  6.6% 12.7%  3.7%
  47. [libx264 @ 0x556669bf9d40] ref B L0: 57.8% 38.6%  3.5%
  48. [libx264 @ 0x556669bf9d40] ref B L1: 96.9%  3.1%
  49. [libx264 @ 0x556669bf9d40] kb/s:49.29
  50.  

I have also a number of attempts that produce direct errors of varying kinds.

This uses a region specifier which works on Windows but not on Linux:
Code: Text  [Select][+][-]
  1. $ ffmpeg -hide_banner -f x11grab -framerate 25 -video_size 622x484 -offset_x 1100 -offset_y 400 -i :0+0,0 -f alsa -i pulse -pix_fmt yuv420p -c:v libx264 -c:a libmp3lame  -q:v 1 -t 20 testvideoA.mp4
  2. Unrecognized option 'offset_x'.
  3. Error splitting the argument list: Option not found
  4.  

Here I tried to use another audio channel because of the mixing of stream and microphone audio:

Code: Text  [Select][+][-]
  1. $ ffmpeg -hide_banner -f x11grab -framerate 25 -video_size 622x484 -i :0+1100,400+nomouse -f alsa -i openal -pix_fmt yuv420p -c:v libx264 -c:a libmp3lame  -q:v 1 -t 20 testvideo10.mp4
  2. Input #0, x11grab, from ':0+1100,400+nomouse':
  3.   Duration: N/A, start: 1614180088.610557, bitrate: N/A
  4.     Stream #0:0: Video: rawvideo (BGR[0] / 0x524742), bgr0, 622x484, 25 fps, 25 tbr, 1000k tbn, 1000k tbc
  5. ALSA lib pcm.c:2642:(snd_pcm_open_noupdate) Unknown PCM openal
  6. [alsa @ 0x562f968b3840] cannot open audio device openal (No such file or directory)
  7. openal: Input/output error
  8.  

I have searched the web and come across some pages that resulted in the command I have that does not error out directly. But I do not yet have anything that does what my windows command does properly...

Is there a way on Linux to capture the desktop image while playing a video in the browser?
I have seen mention of some "privacy" setting in later Linux versions that block applications from reading other application output screens...
Is this what is causing the black screen here?

Edit
I visited this ffmpeg page to check if I could get something going...
From this I got the following command which runs on my Ubuntu box but gives the same two erroneous result:
1) No video, just a black screen
2) Audio from the playing video AND the microphone mixed!
3) I had to set the output file type as mkv (as in the webpage) if I set it to mp4 ffmpg just sits there doing nothing.

Here is what I used to get a (black) video with audio from stream mixed with the microphone:
Code: Text  [Select][+][-]
  1. $ ffmpeg -video_size 854x480 -framerate 25 -f x11grab -i :0.0+100,200 -f pulse -ac 2 -i default -t 10 output2.mk


« Last Edit: February 24, 2021, 06:25:12 pm by BosseB »
--
Bo Berglund
Sweden

BosseB

  • Sr. Member
  • ****
  • Posts: 468
Re: Using ffmpeg to capture video and audio on Ubuntu 20, how to set arguments
« Reply #1 on: February 24, 2021, 07:40:27 pm »
UPDATE
I found a recording application for Linux named Kazam, which I could install via apt.
This application has several different recording modes, among which the screen area mode is selectable too.
When I use this application it does actually record a non-black video, however the audio is silent!
You can select audio source as speaker (which I did) and there is total silence even though the built-in loudspeakers operate normally.
If I instead select Microphone then there is muted sound, apparently using the microphone to record the speaker output via the air...

What a mess...

And I cannot use Kazam anyway because I need to control recordings via my Lazarus application/scheduler...

But it sort of proves that there is no general blocking of access to other application windows...
--
Bo Berglund
Sweden

BosseB

  • Sr. Member
  • ****
  • Posts: 468
Re: Using ffmpeg to capture video and audio on Ubuntu 20, how to set arguments
« Reply #2 on: February 24, 2021, 10:16:01 pm »
Back to ffmpeg...
I am almost there now, can record video from the desktop using the command below and I get audio too.
But the audio comes from the microphone so it is really bad quality, basically the loudspeakers passing through the laptop chassis and mixing with the fan and ambient noise before getting into the mp4 video.
This is what I used now:
Code: Text  [Select][+][-]
  1. $ ffmpeg -hide_banner -f x11grab -framerate 25 -video_size 884x488 -i :0+662,156 -f pulse -i default -preset ultrafast -crf 18 -pix_fmt yuv420p -t 20 output5.mp4

How can I modify it so that it takes the audio from the output to the speaker?

This is how ffmpeg lists the devices on the system:

Code: Text  [Select][+][-]
  1. $ ffmpeg -hide_banner -devices
  2. Devices:
  3.  D. = Demuxing supported
  4.  .E = Muxing supported
  5.  --
  6.  DE alsa            ALSA audio output
  7.   E caca            caca (color ASCII art) output device
  8.  DE fbdev           Linux framebuffer
  9.  D  iec61883        libiec61883 (new DV1394) A/V input device
  10.  D  jack            JACK Audio Connection Kit
  11.  D  kmsgrab         KMS screen capture
  12.  D  lavfi           Libavfilter virtual input device
  13.  D  libcdio          
  14.  D  libdc1394       dc1394 v.2 A/V grab
  15.  D  openal          OpenAL audio capture device
  16.   E opengl          OpenGL output
  17.  DE oss             OSS (Open Sound System) playback
  18.  DE pulse           Pulse audio output
  19.   E sdl,sdl2        SDL2 output device
  20.  DE sndio           sndio audio playback
  21.  DE video4linux2,v4l2 Video4Linux2 output device
  22.  D  x11grab         X11 screen capture, using XCB
  23.   E xv              XV (XVideo) output device
  24.  
--
Bo Berglund
Sweden

Jurassic Pork

  • Hero Member
  • *****
  • Posts: 1228
Re: Using ffmpeg to capture video and audio on Ubuntu 20, how to set arguments
« Reply #3 on: February 25, 2021, 07:34:06 am »
hello,
use pavucontrol utility :
have a look here

Friendly, J.P
Jurassic computer : Sinclair ZX81 - Zilog Z80A à 3,25 MHz - RAM 1 Ko - ROM 8 Ko

BosseB

  • Sr. Member
  • ****
  • Posts: 468
Re: Using ffmpeg to capture video and audio on Ubuntu 20, how to set arguments
« Reply #4 on: February 25, 2021, 01:24:37 pm »
hello,
use pavucontrol utility :
have a look here

Friendly, J.P
Thanks!
After a little fiddling with the pavucontrol application I managed to get it to work.
What I now would like to know is what changes were made to the system by the click actions in the program....
I need to document the findings for future use and it is then best to state what/where changes to the system were made.

Next now is to verify a full recording and then I must get my coordinate finder program to work on Linux.
--
Bo Berglund
Sweden

 

TinyPortal © 2005-2018