Recent

Author Topic: Bug? TFPHTTPClient fails on Android when ConnectTimeout<>0  (Read 4629 times)

engkin

  • Hero Member
  • *****
  • Posts: 3112
Bug? TFPHTTPClient fails on Android when ConnectTimeout<>0
« on: June 11, 2021, 04:54:52 pm »
On Android both in terminal and GUI apps, TFPHTTPClient fails to connect when ConnectTimeout property has a value bigger than zero.

The following project shows the problem:
Code: Pascal  [Select][+][-]
  1. program test_fphttpclient;
  2.  
  3. {$define UseCThreads}
  4. {$mode objfpc}{$H+}
  5.  
  6. uses
  7.   {$IFDEF UNIX}{$IFDEF UseCThreads}
  8.   cthreads,
  9.   {$ENDIF}{$ENDIF}
  10.   Classes,SysUtils,fphttpclient,opensslsockets,openssl,netdb
  11.   { you can add units after this };
  12.  
  13. var
  14.   c:TFPHTTPClient;
  15.   r:RawByteString='';
  16. begin
  17.   WriteLn('GetDNSServers: '+GetDNSServers.ToString);
  18.   WriteLn('InitSSLInterface: '+InitSSLInterface.ToString(TUseBoolStrs.True));
  19.   WriteLn('IsSSLloaded: '+IsSSLloaded.ToString(TUseBoolStrs.True));
  20.  
  21.   c:=TFPHTTPClient.Create(nil);
  22.   try
  23.     c.AllowRedirect:=True;
  24.     c.ConnectTimeout:=0;//<--- Passes
  25.     //c.ConnectTimeout:=3000;//<---- Fails
  26.     try
  27.       r:=c.Get('https://www.google.com');
  28.       WriteLn('Response Length: ',Length(r));
  29.     except
  30.       on e:exception do WriteLn(e.ClassName+': '+e.Message);
  31.     end;
  32.   finally
  33.     c.Free;
  34.   end;
  35. end.
  36.  

Run it using:
Code: Pascal  [Select][+][-]
  1. adb push "$Project()" "/data/local/tmp"
  2. adb shell chmod 755 /data/local/tmp/$ProjFile()
  3. adb shell /data/local/tmp/$ProjFile()
  4.  
  5. OR
  6.  
  7. adb push "$Project()" "/data/local/tmp" && adb shell chmod 755 /data/local/tmp/$ProjFile() && adb shell /data/local/tmp/$ProjFile()

When ConnectTimeout>0, it fails with:
Quote
GetDNSServers: 2
InitSSLInterface: True
IsSSLloaded: True
ESocketError: Connect to www.google.com:443 failed.

Tracking the code:
Code: Pascal  [Select][+][-]
  1. procedure TInetSocket.Connect;
  2. ...
  3.   {$ifdef unix}
  4.   Err:=ESysEINTR;
  5.   While IsError and (Err in [ESysEINTR, ESysEAGAIN]) do
  6.   {$endif}
  7.     begin
  8.     IsError:=fpConnect(Handle, @addr, sizeof(addr))<>0;
  9.     if IsError then
  10.       Err:=Socketerror;
  11.     end;
  12. {$IFDEF HAVENONBLOCKING}
  13.   if (ConnectTimeOut>0) then
  14.     begin
  15.     if IsError and (Err=ErrWouldBlock) then
  16.       begin
  17.       TimeOutResult:=CheckSocketConnectTimeout(Handle, @FDS, @TimeV);
  18.       IsError:=(TimeOutResult<>ctrOK);
  19.       end;
  20.     SetSocketBlockingMode(Handle, bmBlocking, @FDS);
  21.     end;
  22. {$ENDIF}
  23.   If Not IsError then
  24.     begin
  25.     IsError:=Not FHandler.Connect;
  26.     if IsError then
  27.       CloseSocket(Handle);
  28.     end;
  29.   If IsError then
  30.     if TimeoutResult=ctrTimeout then
  31.       Raise ESocketError.Create(seConnectTimeOut, [Format('%s:%d',[FHost, FPort])])
  32.     else
  33.       Raise ESocketError.Create(seConnectFailed, [Format('%s:%d',[FHost, FPort])]);
  34. end;

It fails in the call to fpConnect, but the Err is 0!!!
I don't know what that means.

Is it a bug in the way TInetSocket.Connect handles ConnectTimeout?



As a nice outcome, Android is providing openssl libraries for both the GUI and the terminal apps.



The above test using Laz2.0.10/FPC3.2.0 to cross compile to Android/ArmV7a
« Last Edit: June 11, 2021, 04:58:24 pm by engkin »

engkin

  • Hero Member
  • *****
  • Posts: 3112
Re: Bug? TFPHTTPClient fails on Android when ConnectTimeout<>0
« Reply #1 on: June 13, 2021, 04:46:29 am »
Maybe it is not easy for others to test on their devices.

I saw reports indicating a problem in Android 7, my test device. I can not be sure if the bug is in Android 7, or in TInetSocket.Connect. For now I changed TInetSocket.Connect to include my case:
Code: Pascal  [Select][+][-]
  1. procedure TInetSocket.Connect;
  2. ...
  3.   if (ConnectTimeOut>0) then
  4.     begin
  5.     if IsError and ((Err=ErrWouldBlock) {$ifdef Android}or (Err=0){$endif}) then//<--- is this correct for Android?
  6. ...

LAMW is not using TFPHTTPClient AFAICT.

jmpessoa

  • Hero Member
  • *****
  • Posts: 2296
Re: Bug? TFPHTTPClient fails on Android when ConnectTimeout<>0
« Reply #2 on: June 13, 2021, 07:05:25 pm »
Quote
LAMW is not using TFPHTTPClient AFAICT.

But, you can use TFPHTTPClient and almost all  FCL (not LCL!) in LAMW,  just be careful with the paths and the linking process...

This is a great path to be explored!
« Last Edit: June 13, 2021, 07:11:06 pm by jmpessoa »
Lamw: Lazarus Android Module Wizard
https://github.com/jmpessoa/lazandroidmodulewizard

engkin

  • Hero Member
  • *****
  • Posts: 3112
Re: Bug? TFPHTTPClient fails on Android when ConnectTimeout<>0
« Reply #3 on: June 13, 2021, 07:18:57 pm »
Thank you, jmpessoa.

Can you test this project in LAMW terminal on a real phone?
Code: Pascal  [Select][+][-]
  1. program test_fphttpclient;
  2.      
  3.     {$define UseCThreads}
  4.     {$mode objfpc}{$H+}
  5.      
  6.     uses
  7.       {$IFDEF UNIX}{$IFDEF UseCThreads}
  8.       cthreads,
  9.       {$ENDIF}{$ENDIF}
  10.       Classes,SysUtils,fphttpclient,opensslsockets,openssl,netdb
  11.       { you can add units after this };
  12.      
  13.     var
  14.       c:TFPHTTPClient;
  15.       r:RawByteString='';
  16.     begin
  17.       WriteLn('GetDNSServers: '+GetDNSServers.ToString);
  18.       WriteLn('InitSSLInterface: '+InitSSLInterface.ToString(TUseBoolStrs.True));
  19.       WriteLn('IsSSLloaded: '+IsSSLloaded.ToString(TUseBoolStrs.True));
  20.      
  21.       c:=TFPHTTPClient.Create(nil);
  22.       try
  23.         c.AllowRedirect:=True;
  24.         c.ConnectTimeout:=3000;
  25.         try
  26.           r:=c.Get('https://www.google.com');
  27.           WriteLn('Response Length: ',Length(r));
  28.         except
  29.           on e:exception do WriteLn(e.ClassName+': '+e.Message);
  30.         end;
  31.       finally
  32.         c.Free;
  33.       end;
  34.     end.

jmpessoa

  • Hero Member
  • *****
  • Posts: 2296
Re: Bug? TFPHTTPClient fails on Android when ConnectTimeout<>0
« Reply #4 on: June 13, 2021, 11:33:45 pm »

My limited test:  (I need update my lazarus/fpc....)

Quote
06-13 16:56:34.925 24502-24502/org.lamw.apptfphttpclientdemo1 I/ShowMessage: InitSSLInterface: True
06-13 16:56:34.955 24502-24502/org.lamw.apptfphttpclientdemo1 I/ShowMessage: IsSSLloaded: True
06-13 16:56:34.965 24502-24502/org.lamw.apptfphttpclientdemo1 D/LAMW: ESocketError: Host name resolution for "www.google.com" failed.
Lamw: Lazarus Android Module Wizard
https://github.com/jmpessoa/lazandroidmodulewizard

engkin

  • Hero Member
  • *****
  • Posts: 3112
Re: Bug? TFPHTTPClient fails on Android when ConnectTimeout<>0
« Reply #5 on: June 14, 2021, 01:09:45 pm »
Yes, you are having an older bug. Without updating you may be able to pass this bug by using Google DNS servers:
Code: Pascal  [Select][+][-]
  1. begin
  2.   SetLength(DNSServers, 2);
  3.   DNSServers[0]:=StrToNetAddr('8.8.8.8');
  4.   DNSServers[1]:=StrToNetAddr('8.8.4.4');
  5. ...
« Last Edit: June 14, 2021, 01:12:24 pm by engkin »

jmpessoa

  • Hero Member
  • *****
  • Posts: 2296
Re: Bug? TFPHTTPClient fails on Android when ConnectTimeout<>0
« Reply #6 on: June 14, 2021, 06:38:18 pm »
The same... [Without updating...]
Quote
06-14 12:31:37.576 6751-6751/org.lamw.apptfphttpclientdemo1 I/ShowMessage: InitSSLInterface: True
06-14 12:31:37.606 6751-6751/org.lamw.apptfphttpclientdemo1 I/ShowMessage: IsSSLloaded: True
06-14 12:31:47.626 6751-6751/org.lamw.apptfphttpclientdemo1 D/LAMW: ESocketError:Host name resolution for "www.google.com" failed.

My code:

Code: Pascal  [Select][+][-]
  1. {hint: Pascal files location: ...\AppTFPHTTPClientDemo1\jni }
  2. unit unit1;
  3.  
  4. {$mode delphi}
  5.  
  6. interface
  7.  
  8. uses
  9.   {$IFDEF UNIX}{$IFDEF UseCThreads}
  10.   cthreads,
  11.   {$ENDIF}{$ENDIF}
  12.  
  13.   Classes, SysUtils, AndroidWidget, Laz_And_Controls,
  14.   fphttpclient,
  15.   httpdefs,
  16.   httpprotocol,
  17.   //opensslsockets,
  18.   openssl, Sockets,
  19.   netdb;
  20.  
  21. type
  22.  
  23.   { TAndroidModule1 }
  24.  
  25.   TAndroidModule1 = class(jForm)
  26.     jButton1: jButton;
  27.     procedure jButton1Click(Sender: TObject);
  28.   private
  29.     {private declarations}
  30.   public
  31.     {public declarations}
  32.   end;
  33.  
  34. var
  35.   AndroidModule1: TAndroidModule1;
  36.  
  37. implementation
  38.  
  39. {$R *.lfm}
  40.  
  41.  
  42. { TAndroidModule1 }
  43.  
  44. //C:\laz4android2.0.0\fpc\3.0.4\source\packages\fcl-web\src\base  <--- (-Fu)
  45. procedure TAndroidModule1.jButton1Click(Sender: TObject);
  46. var
  47.   c:TFPHTTPClient;
  48.   r:RawByteString='';
  49. begin
  50.  
  51.    SetLength(DNSServers, 2);
  52.    DNSServers[0]:=StrToNetAddr('8.8.8.8');
  53.    DNSServers[1]:=StrToNetAddr('8.8.4.4');
  54.  
  55.    ShowMessage('InitSSLInterface: ' + InitSSLInterface.ToString(TUseBoolStrs.True));
  56.    ShowMessage('IsSSLloaded: ' +      IsSSLloaded.ToString(TUseBoolStrs.True));
  57.  
  58.   c:=TFPHTTPClient.Create(nil);
  59.   try
  60.     c.AllowRedirect:=True;
  61.     //c.ConnectTimeout:=3000;
  62.     try
  63.       r:=c.Get('https://www.google.com');
  64.       ShowMessage('Response Length: '+IntToStr(Length(r))); // NOT CALLED !!!
  65.     except
  66.       on e:exception do LogDebug('LAMW',e.ClassName+':'+e.Message);
  67.     end;
  68.   finally
  69.     c.Free;
  70.   end;
  71.  
  72. end;
  73.  
  74. end.
  75.  
Lamw: Lazarus Android Module Wizard
https://github.com/jmpessoa/lazandroidmodulewizard

engkin

  • Hero Member
  • *****
  • Posts: 3112
Re: Bug? TFPHTTPClient fails on Android when ConnectTimeout<>0
« Reply #7 on: June 14, 2021, 07:29:08 pm »
I took the relevant files and changed their names so they don't conflict with the original files in your installation, did some logging. You can change the procedure in ulog unit if you need to.

Try it with and without -dPossibleFix in the Custom Options.

As the last line when it fails without -dPossibleFix, I expect you to see:
ESocketError: Connect to www.google.com:443 failed.

With -dPossibleFix it should give you something along:
Response Length: 49123

 

TinyPortal © 2005-2018