Tool to extract m3u8 stream from embedded video player?

[BosseB]

But there is another problem this script does not handle:
Pages with multiple video players...
It is not very common but my the page I used back in 2018 suddenly stuffed a second player on the same page and I guess this would need to be handled. That page is long gone, though, so currently no problem like that.

Theoretically, there should be one video source, and there may be several players, but they all use the same link to the stream, or is it not so?

That site is decommissioned, just mentioned it as a possible problem when extracting the m38u url...

I have another site where I cannot get the m3u8 at all, not using my extractors or the debug view (F12) or even the VideoController extension in Firefox. Nothing works for this site...

[delphius]

I have another site where I cannot get the m3u8 at all, not using my extractors or the debug view (F12) or even the VideoController extension in Firefox. Nothing works for this site...

It looks like this is a live broadcast of the twitch streaming service, try to work in this direction, and here is a similar utility for YouTube.

[BosseB]

I have another site where I cannot get the m3u8 at all, not using my extractors or the debug view (F12) or even the VideoController extension in Firefox. Nothing works for this site...

It looks like this is a live broadcast of the twitch streaming service, try to work in this direction, and here is a similar utility for YouTube.

I am using yt-dlp to download static video files and to check the available resolutions in a video.
But I don't know how to make it work for a streaming service where one needs to specify also the download duration. Seems like yt-dlp is not capable of downloading from a stream...
If it could start downloading from time now and then run until being stopped by a Ctrl-C or similar and when it does stop then finish the downloaded file so it is playable, then it could perhaps work.
But is this possible? And if so how? Remember that the problem is finding the stream URL, which I have failed to do for this site.

[TRon]

But is this possible? And if so how? Remember that the problem is finding the stream URL, which I have failed to do for this site.

Sure but this is becoming close to not being legal (depends on country).

If you want a hint then have a look at these instructions (just checked, these instructions work for your example stream).

If you can do that stuff manually then you can also do it automated (and has been done so by many tools out there that are available).

[BosseB]

But is this possible? And if so how? Remember that the problem is finding the stream URL, which I have failed to do for this site.

Sure but this is becoming close to not being legal (depends on country).

If you want a hint then have a look at these instructions (just checked, these instructions work for your example stream).

If you can do that stuff manually then you can also do it automated (and has been done so by many tools out there that are available).

Thanks, I persisted in following the instructions in your linked page and got the m3u8 URL for the stream. 

It turns out to be a 998 char long m3u8 URL where the content seems to be Base64 encoded, but it won't decode as such so it is some other kind of m3u8 encryption.
But I put it as-is into my download script for the source and made a 30 second download using it.
This seemed to be exactly like I want it to be.
So now I am doing a complete 1 hour download using this m3u8.
Interesting to see when/if it will fail.

The question now is how to extract the m3u8 url programmatically (if it expires regularly, if not then it probably will work into the future).

[TRon]

It turns out to be a 998 char long m3u8 URL where the content seems to be Base64 encoded, but it won't decode as such so it is some other kind of m3u8 encryption.

huh ? If I download that m3u8 URI with wget I get a 'normal' text file that represents a m3u file (with video related extensions ofc). Did you use/do something special to download the m3u8 file for that URL ?

So now I am doing a complete 1 hour download using this m3u8.

You should be able to automate that, but depends a little on the download utility that you use. I assume that you are only interested in 'some' parts of that live stream (for example the regular weather report that is aired at specific times/time intervals) in which case you should be able to write a small utility that runs in the background, checking the date/time (intervals) and invoke the recording utility for a specific amount of time (or kill/close it's process ID when that is not possible).

The question now is how to extract the m3u8 url programmatically (if it expires regularly, if not then it probably will work into the future).

Usuaulyl sites suh as twitch that display those videos have a API that you can use to extract all kinds of information. Some API's are free, some paid, other sometimes require an account. I do not know anything about Twitch so I have no idea.

I did came across this utility that has some interesting comments on its twitch plugin (be sure to read that, also to understand better why it is not always legal to do as you request and platforms like twitch tries to protect itself against such practices).

Also as a hint, make sure to do a search for twitch and record/save as there are many other such kind of tools like the utility mentioned above (also those that use twitch API to extract all kinds of information from a stream, most come with sourcecode, hint: include github (or any other source hosting sites) to your searches  ).

[Fibonacci]

I checked this webui lib you linked to cos Im interested in alternatives to LCL. It looks nice, I was even able to static-compile this lib into my app so no DLL file needed. Single exe file, 261 KB.

It's great that you managed to try the library and link it statically. Could you write a little instruction on how to do this for other users?

Already replied to you PM but I think more people may be interesed in static compiling so I'll include source in the attachment.

These files you need to find yourself:
{$linklib libkernel32.a}
{$linklib libadvapi32.a}
{$linklib libuser32.a}
{$linklib libmingwex.a}
{$linklib libucrt.a}
{$linklib libmincore.a}

The question now is how to extract the m3u8 url programmatically (if it expires regularly, if not then it probably will work into the future).

Obviously it will expire (its a valid assumption). Also you try to decode that base64, why? Dont do it, its some kind of auth token, you shouldnt care about it. Just get valid URL and use it, if it expires then extract a new one.

[BosseB]

The question now is how to extract the m3u8 url programmatically (if it expires regularly, if not then it probably will work into the future).

Obviously it will expire (its a valid assumption). Also you try to decode that base64, why? Dont do it, its some kind of auth token, you shouldnt care about it. Just get valid URL and use it, if it expires then extract a new one.

So I have made some more tests now using the FireFox F12 debug window to get the m3u8 url to use with the ffmpeg download.
I have discovered this:

1) The main site url will display a player but sometimes the player itself shows a message to "refresh the browser" instead of the actual video. Refreshing the browser does not do anything...

2) In this situation there is below the player two links "STREAM-1" and "STREAM-2" and STREAM-1 seems to be the default which displays the message above.

3) By hitting STREAM-2 the video starts playing and it shows the expected content but with a different overlay from the other stream (when it worked). This stream is less reliable than the original, though. It refreshes visibly now and then...

4) The lifetime of the m3u8 url extracted using the FireFox F12 tool seems to have a lifetime of about a day, so the automation of extraction would help here. But when now dealing with two streams complicates things a lot...

I am using this source as a backup only so if the standard source works it is not used, only when that fails.

[Fibonacci]

It displays "PLEASE REFRESH THE BROWSER" even in thumbnail

https://www.twitch.tv/solanna89

[Fibonacci]

Stream 2 works, so

1. Go to https://www.twitch.tv/grhntlk
2. Open devtools, F12
3. Reload
4. Search for "ttvnw.net/v1/playlist"
5. You will get a bunch of links, get the first one
6. Copy it and open in VLC
7. Works, you can pass this link to ffmpeg or ytdl

How to do it in pascal? I wouldn't even bother. Just allow it to process all javascript and catch first URL to m3u8. Use browser extension or webdriver

[BosseB]

Stream 2 works, so

1. Go to https://www.twitch.tv/grhntlk
2. Open devtools, F12
3. Reload
4. Search for "ttvnw.net/v1/playlist"
5. You will get a bunch of links, get the first one
6. Copy it and open in VLC
7. Works, you can pass this link to ffmpeg or ytdl

How to do it in pascal? I wouldn't even bother. Just allow it to process all javascript and catch first URL to m3u8. Use browser extension or webdriver

For me it does not display anything but an "offline" message:

(how can you make the attachment image display inside the post itself rather than as a listed attachment?)

[Fibonacci]

For me it does not display anything but an "offline" message:

At this moment its actually offline Test with other stream

(how can you make the attachment image display inside the post itself rather than as a listed attachment?)

I dont think you can

[delphius]

I think more people may be interesed in static compiling so I'll include source in the attachment.

Slightly modified, taking into account the changes in the static version of the library in August (+$linklib libs/libmsvcr120.a)
Posted here to make it convenient to test

[TRon]

How to do it in pascal? I wouldn't even bother. Just allow it to process all javascript and catch first URL to m3u8. Use browser extension or webdriver

Another possible solution (next to the already mentioned stream tools) is to use something like youtube_dl (search for the working URL as the original is down after court ruling)

Code: [Select]

youtube-dl -g https://www.twitch.tv/"nameofchannel/livestream"
Which will then return the link to the m3u8 file for you.

fwiw: as suspected the lokinau link is down for reasons mentioned before.