[Solved] TFPHTTPClient.Head/Get(URL, SL/MS) problems.

[Fred vS]

Hello!

It is about dealing with web audio radios.

First the good news.
I have success with those code, Http.Get() is ok, the pipe is working and the audio is ok:

Code: Pascal  [Select][+][-]

1. var
2. FOutStream: TOutputPipeStream;
3. Http: TFPHTTPClient;
4. ...
5.   InitSSLInterface;
6.   Http := TFPHTTPClient.Create(nil);
7.   try
8.     Http.AllowRedirect := True;
9.     Http.IOTimeout := 5000;
10.     Http.RequestHeaders.Clear;
11.     Http.Get(URL, FOutStream);
12. ...

With this code, a pre-init to get the info of the stream, most of time Http.Head() works, SL is filled and if not accepted by the stream, it fast gives a error so I can deal with it.

Code: Pascal  [Select][+][-]

1. var
2. SL: TStringList;
3. Http: TFPHTTPClient;
4. ...
5.   InitSSLInterface;
6.   Http := TFPHTTPClient.Create(nil);
7.   SL   := TStringList.Create;
8.   try
9.     Http.AllowRedirect := True;
10.     Http.IOTimeout := 5000;
11.     Http.RequestHeaders.Clear;
12.     Http.Head(URL, SL);
13. ...

But with this code, a other pre-init to check the info of the stream if Http.head() has failed, Http.Get() take lot of time to get, it is like it is sleeping.

Code: Pascal  [Select][+][-]

1. var
2. TempStream := TMemoryStream;
3. Http: TFPHTTPClient;
4. ...
5.   InitSSLInterface;
6.   Http := TFPHTTPClient.Create(nil);
7.   TempStream := TMemoryStream.Create;
8.   TempStream.Position := 0;
9.    try
10.     Http.AllowRedirect := True;
11.     Http.IOTimeout := 5000;
12.     Http.RequestHeaders.Clear;
13.     Http.AddHeader('Range', 'bytes=0-2047'); // Fetch only 2 KB
14.     Http.Get(URL, TempStream);
15.   ...

Tested with fpc 3.2.2. and 3.3.1-trunk.
 
How can I fix this and if you have a other idea how to get the infos of the stream it will be welcome.

Fre;D

[Leledumbo]

As the only difference with the very first code is the existence of Range header, you need to ensure the server supports that. With either HEAD or GET, inspect the response header, it should have:

Code: [Select]

Accept-Ranges: bytes
If not, assume the server doesn't support and server behavior is undefined if given. fphttpclient should have complete support for HTTP 206, at least in trunk.

[Fred vS]

As the only difference with the very first code is the existence of Range header, you need to ensure the server supports that. With either HEAD or GET, inspect the response header, it should have:

Code: [Select]

Accept-Ranges: bytes
If not, assume the server doesn't support and server behavior is undefined if given. fphttpclient should have complete support for HTTP 206, at least in trunk.

Thanks Leledumbo for the answer.
Hum, so using Range header would not be a "universal" solution.

I would prefer, because Http.Head() does not work for every web radio (but using curl, I get the info-header), to use the main Http.Get(URL, FOutStream) that works well to sent the audio pipe to get the info-header.
Maybe catch a initial buffer of the pipe and analyze it?

[EDIT] For example, that audio stream accept HEAD using curl but not fpc:

Code: Pascal  [Select][+][-]

1. $ curl -i https://scdn.nrjaudio.fm/adwz1/fr/30663/mp3_128.mp3
2. HTTP/1.1 302 D�plac� Temporairement
3. Server: Apache-Coyote/1.1
4. Access-Control-Allow-Headers: *
5. Access-Control-Allow-Methods: HEAD, GET, POST, OPTIONS
6. Access-Control-Allow-Origin: *
7. Set-Cookie: JSESSIONID=8AD6E462DB43A4F3A4BFC68DDE6125D8; Path=/; HttpOnly
8. Location: https://scdn.nrjaudio.fm/fr/30663/mp3_128.mp3?cdn_path=adswizz_lbs7&adws_out_1&access_token=d66e63a39f414355bbdd832021687688
9. Content-Length: 0
10. Date: Sat, 12 Apr 2025 05:53:07 GMT
11. Strict-Transport-Security: max-age=16070400

But this is ok for fpc to use head:

Code: Pascal  [Select][+][-]

1. curl -i https://radiorecord.hostingradio.ru/sd9096.aacp
2. HTTP/1.1 200 OK
3. Content-Type: audio/aac
4. Date: Sat, 12 Apr 2025 05:58:24 GMT
5. icy-description:by Radio Record
6. icy-genre:Dance
7. icy-metadata:1
8. icy-name:SuperDiskoteka90
9. icy-pub:1
10. icy-url:http%3a%2f%2fradiorecord.ru
11. Server: Icecast 2.4.0-kh10-hss7
12. Cache-Control: no-cache, no-store
13. Expires: Mon, 26 Jul 1997 05:00:00 GMT
14. Connection: Close
15. Set-Cookie: hssuid=f135b555cc3046409efe83db8204626c; expires=Sat, 12-Apr-2026 05:58:24 GMT; domain=.hostingradio.ru; path=/; SameSite=None; Secure
16. Access-Control-Allow-Origin: *
17. Access-Control-Allow-Headers: Origin, Accept, X-Requested-With, Content-Type, Icy-MetaData
18. Access-Control-Allow-Methods: GET, OPTIONS, SOURCE, PUT, HEAD, STATS
19.  

[Leledumbo]

For example, that audio stream accept HEAD using curl but not fpc:

Unfortunately, Head implementation is hardcoded to accept only HTTP 200, any other response code even if it's a redirect is not allowed (even when AllowRedirect is set to true!).
To be fair, this code is OLD (2013, with KeepConnection := false; added in 2016), so it wasn't really well designed.

You can of course just copy the implementation (like every other methods, it just wraps the call to HTTPMethod) allowing 302 to be accepted also.

[Fred vS]

You can of course just copy the implementation (like every other methods, it just wraps the call to HTTPMethod) allowing 302 to be accepted also.

Yeeep, this does the trick:
     

Code: Pascal  [Select][+][-]

1.       Http.KeepConnection := False;
2.       Http.HTTPMethod('HEAD', URL, Nil, [200, 204, 301, 302, 303, 307, 308, 400]);
3.       SL.Assign(http.ResponseHeaders);

Many, many thanks Leledumbo and have a perfect day.

Fre;D