FFGrab4Laz = How to record with FFmpeg and Lazarus

[metis]

@ALL, specially @Mike.Cornflake

'FFPlay4Laz' (= play MediaFiles with FFmpeg-Libraries and Lazarus)
http://forum.lazarus.freepascal.org/index.php/topic,26666.0.html
and
'FFInfo4Laz' (= read out MediaFile-Data with FFmpeg-Libraries and Lazarus)
http://forum.lazarus.freepascal.org/index.php/topic,22038.msg208700.html#msg208700
are on the Way.

Now, let's do Recording with FFmpeg and Lazarus - I call this Project FFGrab4Laz.

As a Starter, I wrote two small Apps, which grab the Desktop/Screen with FFmpeg, see
attached 'FFGrab4Laz_01.7z'.
(written with Lazarus v1.8.2 + FPC v3.0.4 and FFmpeg v3.4.2 on WinXP 32bit SP3)

> 'ScreenRecorder2FPC'
is a Console, that grabs the entire Screen to a VideoFile (Preset=25 Frames) with FFmpeg-LIBs.
It's my FPC-Port of the 'Screen recorder':
https://github.com/abdullahfarwees/screen-recorder-ffmpeg-cpp.
The Port is kept as closely as possible to the original C-Code to make it easier to compare with FPC,
just as I did with the Dranger-C-Files (= Decoding&Playing with FFmpeg-LIBs) some Years ago:
http://forum.lazarus.freepascal.org/index.php/topic,26666.msg196377.html#msg196377.
Adaptations and Modifications are denoted as "by moi", so that You can find them easily.

To compile it, You have to download the FFmpeg-PascalHeaders from
http://www.delphiffmpeg.com/headers/.
-> Download 'ffmpeg-delphi-pascal-headers-3.4.2.zip' and copy them into Dir [Libs\ffmpeg-3.4.2].

To run it, You have to download the FFmpeg-DLLs, e.g. from
http://www.delphiffmpeg.com/downloads/ ("Encoder and Player"), and
put them into the same Directory as the compiled Binary, that is [bin_ffmpeg-3.4.2\win32].

For a (Delphi)Pascal-Sample to encode Video and Audio with FFmpeg-LIBs,
open the DelphiProject "..\examples\muxing.dpr" in the downloaded FFmpeg-PascalHeaders-File.


> 'RunFFmpeg'
is a small Tool to check & run the FFmpeg Command Line Tools (= FFmpeg-CLIs):
http://ffmpeg.org/about.html -> GoTo "FFmpeg Tools".
It uses the Lazarus-TProcess-Component and is derived from the Lazarus-ProcessDemo:
http://wiki.lazarus.freepascal.org/Executing_External_Programs -> GoTo "See processdemo example".
(The Link does not work any more; therefore I've put the ProjectFiles into Dir [ProcessDemo] of the attached 7z-File.)

The DemoVersion (see attached 'RunFFmpegDemo.jpg') has preset CommandLines, to
grab any Part of the Screen to a VideoFile (= [Grab]), then check (= [Info]) and play (= [Play]) the File.
If You parse the FFmpeg-ProcessOutput, You can read out whatever You want.
If You change the CommandLines, You can do whatever FFmpeg is able to do, e.g.:
[Grab]+[-i "grabbed.mkv" -map v frame%08d.png] will save the File's VideoFrames to a Series of PNG-ImageFiles.

'RunFFmpeg' requires 'ffmpeg.exe', 'ffprobe.exe' and 'ffplay.exe', which can be downloaded e.g. from
http://ffmpeg.zeranoe.com/builds/ ("static Builds" or better "shared Builds and DLLs").
To play the recorded File with 'ffplay.exe', You need 'SDL2.dll', too:
http://www.libsdl.org/download-2.0.php -> GoTo "Runtime Binaries".
Put all downloaded Files into the same Directory [bin_ffmpeg-3.4.2\win32].
Use the '-WG Switch' to open 'RunFFmpeg' with or w/o ConsoleWindow.

After Downloading and Compiling the ProjectFiles,
Your FFGrab4Laz-Directory should look as shown in attached 'FFGrab4LazFiles.jpg'.

For how to build FFmpeg-CommandLines in General, see the FFmpeg-Documentation:
http://ffmpeg.org/documentation.html -> GoTo "Command Line Tools Documentation".

For how to build FFmpeg-CommandLines with FPC, see the SourceFiles of 'WinFF', that is
an Open-Source MediaConverter, written with FPC, which uses 'ffmpeg.exe' and 'ffplay.exe' with 'TProcess':
https://www.biggmatt.com/winff/.

The FFmpeg-CommandLine-Arguments for Recording are the same as for Transcoding, only the Input changes.

 

[Mike.Cornflake]

Nice work, many thanks

[metis]

Hint:
'RunFFmpeg' is not only for Grabbing. It does anything You want 'FFmpeg' to do for You.

Here is an Example for Streaming with 'RunFFmpeg' on Win10:
Put http://tagesschau-lh.akamaihd.net/i/tagesschau_1@119231/index_1992_av-p.m3u8?sd=10&rebase -x 950 -y 600 -window_title " This is my window title"
into the Line next to [Play] and click [Play].
 

[metis]

And here is an Example to convert a MPEG-Video to a h264/AAC mp4-File with 'RunFFmpeg':
Put -i "input.mpg" -c:v libx264 -b:v 800k -c:a aac -b:a 128k "output.mp4"
into the Line next to [Grab] and click [Grab].

Or the same Conversion with some more Arguments (StreamMapping, Scaling, AspectRatio, AudioPadding):
-i "input.mpg" -map 0:0 -filter:v:0 "scale=1280:720" -c:v:0 libx264 -b:v:0 800k -aspect:v:0 16:9 -map 0:1 -c:a:0 aac -b:a:0 160k -shortest "output.mp4"

 

[metis]

In the Meantime, there were two Questions in my FFPlay4Laz-Thread, which actually should have been posted in this Thread:

Is it possible to use this project to create screen recording software?

see https://forum.lazarus.freepascal.org/index.php/topic,26666.msg373412.html#msg373412

and

I want to create a video/audio capture app with Lazarus/Fpc

see https://forum.lazarus.freepascal.org/index.php/topic,26666.msg382591.html#msg382591

I answered to both Questions there, even though I didn't want to merge those two Subjects, because
the Approaches are quite different.

[metis]

The Following is my Answer to this and this, again wrongly posted in my FFPlay4Laz-Thread...

@BosseB

Is it needed to use a 32 bit Lazarus/Fpc as mentioned in descriptions?

No, the SourceCode of 'RunFFmpeg' should compile to 64-bit, too.

What are the caveats when not using 64 bit?

There, aren't. With Win10, You can use both, 64-bit and 32-bit Programs.

the ffmpeg binaries are not found. But they are on my path

They are not. You've put them into the Folder, where Programs are typically installed on Windows.
The FFmpeg-CLIs (= CommandLine Interfaces) need no Installation (unlike 'VLC').
You simply put them, where You need them.

ffmpeg will be used as found on the path

Generally, Windows searches for a File in the following Sequence, when no Path is specified:
1. The Directory from which the Program loaded (<- This is, what I describe.)
2. The current Directory
3. The Windows SystemDirectories (= [C:\WINDOWS\system], [C:\WINDOWS\system32])
4. The Windows Directory (= [C:\WINDOWS])
5. The Directories, that are listed in the PATH EnvironmentVariable. (<- This is, what You tried.)
You may put the FFmpeg-CLIs wherever You want. For me, the first Option is the easiest one.

desktop caption ... (Chrome playing a YouTube video)

You can only record or the entire Screen or a Region or an App's Window.
Your CommandLine tries to tell 'FFmpeg' to record a Region (815x443 at 60/200) and the Firefox-Window at the same Time.
How should this work ?
Apart from this, there is no Output-File specified - So, where should 'FFmpeg' record to ?
Last but not least, Your CommandLine has no Arguments to record Audio, neither.

I wonder if there is not a possibility to define the source window using its handle instead?

Never tried that - Check the FFmpeg-Documentation, if WindowHandles are allowed, too:
http://www.ffmpeg.org/ffmpeg-all.html.

Otherwise is there a flag to tell ffmpeg to use a partial match for the title?

Never tried that - Check the FFmpeg-Documentation for that, too.

And where is the grabbed video file supposed to wind up?

If the Output-File has no a Path (like "grabbed.mkv"), it will be created in the same Directory, where the 'ffmpeg.exe' is.

What do I do then???

First of all: Read and follow my Instructions for how to use 'RunFFmpeg'.

Summary:

Try this FFmpeg-CommandLine with RunFFmpeg's [Grab]:
-f gdigrab -framerate 25 -offset_x 60 -offset_y 200 -video_size 815x443 -show_region 1 -i desktop -f dshow -i audio="Sound Blaster Audigy" -vcodec libx264 -pix_fmt yuv420p -preset ultrafast "grabbed.mkv",
where You have to replace "Sound Blaster Audigy" with Your Soundcard's Name.

To get Your Soundcard's Name, use 'ffmpeg.exe' directly:
(May not work with 'RunFFmpeg' - don't know, why.)
ffmpeg -list_devices true -f dshow -i dummy -report

And for Details about a particular Device (here "Your Soundcard"), run this:
ffmpeg -f dshow -list_options true -i audio="Your Soundcard" -report

Both create a LogFile in the same Directory, where You run 'ffmpeg.exe', which contain
all Infos about AudioDevices, that are installed on Your System (and supported by 'FFmpeg').

Hope, I could help You. 

[BosseB]

If I use ffmpeg to extract  from a specific time into the video for a set duration, both in seconds, then the resulting video does not really start at the point I wanted.
It seems to always contain something before the clip start point, it can be 1-2 seconds but sometimes longer than that.
And the resulting video is not "clean" there are very often some kind of noise squares before the image stabilizes.
How come this happens?
Are mp4 videos quantized such that there is a video "atom" which cannot be divided and any clip will be done at "atom" borders?
I figured it should be possible to cut at frame borders (1/25 or 1/30 s) but it does not seem like this is happening.
The command I use to extract the tmp files is like this example (here the start is 1250s into the video and lasts for 600 s):

Code: Text  [Select][+][-]

1. ffmpeg -ss 1250 -i input.mp4 -to 600 -c copy output.mp4

I have searched the net for some solution and found this post on StackOverflow.
Here I found this command, wich extracts multiple sections instead of one like I do with the command above:

Code: Text  [Select][+][-]

1. ffmpeg -i video \
2.        -vf "select='between(t,4,6.5)+between(t,17,26)+between(t,74,91)',
3.             setpts=N/FRAME_RATE/TB" \
4.        -af "aselect='between(t,4,6.5)+between(t,17,26)+between(t,74,91)',
5.             asetpts=N/SR/TB" out.mp4

When I try this then the cut is clean but takes a very long time to do (minutes versus seconds).
Is there a way to make one cut quicker like the first command above and still get a clean result?

[metis]

@BosseB

If I use ffmpeg to extract..., both in seconds

I'm a bit astonished, that "Seconds" are precise enough for Editing Movies.

It seems to always contain something before the clip start point

Take a Look at the FFmpeg-Documentation:
"Note that in most formats it is not possible to seek exactly, so ffmpeg will seek to the
 closest seek point before position. ... When doing stream copy...it will be preserved."
-> Did You try with -accurate_seek ?

Are mp4 videos quantized such that there is a video "atom" which cannot be divided

Every Video is a Series of Images with Timestamps (and other Data) to get them decoded in the right Order and synchronized.
For better Understanding, You may take a little Internet-Trip, looking for:
DTS (= Decoding Timestamp), PTS (= Presentation Timestamp), TB (= Timebase),
I-Frames (= intra-coded Frames), P-Frames (= predicted Frames), B-Frames (= bi-directionally predicted Frames),
GOP (= Group of Pictures) and Keyframes.

When I try this then the cut is clean but takes a very long time to do

setpts and asetpts change the PTS of the Video-/AudioFrames.
Recalculating the PTSs all over the Movie sounds quite time-consuming.
Suppose, that's not the best Solution.

Keep on Trying and tell me/us. 

[metis]

@BosseB

Just now found for You...

Top10 Commands for Lossless Video Manipulation using ffmpeg:
https://forum.videohelp.com/threads/398507-Top10-Commands-for-Lossless-Video-manipulation-using-ffmpeg-%28Guide%29
-> GoTo "Extract a segment from a video (cut on keyframes)"
-> GoTo "Split a video into multiple parts based on duration (ex: 20min segments, cut on keyframes)"
Try it out and tell me/us. 

Hint: The VideoHelp-Site is full of useful Infos & Tutorials about VideoProcessing. 

[BosseB]

On request I moved this question over here...

@metis:
I am capturing video and audio from a browser playing a broadcast stream and it works well after I have applied a number of your suggestions to the ffmpeg command I use.

What I find is that the audio sync is off by about 650 ms, so I have to apply a second ffmpeg command to the download to correct the time shift (I apply -0.65 as the time shift).

Is there a way to apply this already at the capture stage, i.e. by adding the -itsoffset -0.65 to the download command itself?
I tried it without really knowing how to apply it (just added to the end before the capture time and output file), and it failed...

This is how I do it after download:

Code: Text  [Select][+][-]

1. ffmpeg -hide_banner  -i input.mp4 -itsoffset -0.65 -i input-mp4 -map 1:v -map 0:a -c copy output.mp4

This is what I tested to do it during the download but without success (unrecognized option itoffset):

Code: Text  [Select][+][-]

1. ffmpeg -hide_banner -f gdigrab -framerate 25 -offset_x 1042 -offset_y 228 -video_size 854x480 -i desktop  -f dshow -i audio="virtual-audio-capturer" -vcodec libx264 -pix_fmt yuv420p -itoffset -0.65 -t 3600 testaudiodelay.mp4

I am now on 3 different computers trying to move the capture part to Linux, but so far I only get black video (for another thread)...
I used to download and then process videos from Youtube, but now the source I used has disappeared so I must download in real time on a transmission that cannot be interfaced directly AFAIU.
So I need screen capture, which is now going in Windows.
I would like to get it going on Ubuntu too, but unfortunately it looks like capturing video from windows owned by other applications like Firefox seems to be impossible...
So I am still on Windows.

[metis]

@BosseB

to do it during the download but without success

For what You want to do, the recorded File is the Input, which
then can be processed to an other OutputFile. I didn't try that out, but it should work, because
in general several FFmpeg-Tasks can be combined in one CommandLine (with more CPU-Load, of course).

unrecognized option itoffset

Means, that the FFmpeg-Option does not support this Argument.
You've put it at a wrong Place in the FFmpeg-CommandLine.

trying to move the capture part to Linux, but so far I only get black video

'gdigrab' is a VideoDriver for Windows; for LINUX You should use one of these, listed under "LINUX".

[metis]

I've posted 'RunFFmpeg' together with some useful FFmpeg-CommandLines in the VideoHelp-Forum:

https://forum.videohelp.com/threads/400709-GUI-for-ffmpeg#post2611183.

[BosseB]

@metis,
thanks for your hint about the VideoHelp forum, I went there and received help on my questions about ffmpeg possibilities.
Very useful place I did not know about until you mentioned it!

Especially useful is the ffmpeg command to download from a stream playing in a player on a webpage while modifying the video size and compression on the fly! Here is the reply I got.
Very useful and time saving.

[metis]

@BosseB

Here is the reply I got.

Perfect !
Best Site I know, if You're Looking for CommandLine- or Script-Solutions.
I found it, when I searched for FFmpeg-CommandLines 4 u.

Just wondering, where You always get this "340 ms difference" from.
This would mean a quite long AudioLatency with Your System.

[BosseB]

I am also wondering, but it might be caused by my Ubuntu 18.04 server where the download is being executed on. It is running on hardware I inherited from my mother when she died in 2016. It is an eMachines mini-tower that originally came with Windows7. Probably not a powerhouse...

Before I got the reply from the VideoHelp forum on how to download directly from the stream I had set up an old laptop with Ubuntu 20.04 to run screen capture to get the videos overnight.
It worked but meant I had to have a desktop computer running with a browser playing the stream 24/7, which is not optimal. And that player had to be precisely located on screen so the capture could grab the correct data.

Anyway on that setup the audio/video lag was worse. Upwards of 800-900 ms.
Now I get 350-400 ms on the mini-tower (Ubuntu installation without any GUI parts, it is a server).

So it is probably caused by the CPU load...