Recent

Author Topic: how to speed up fphttpclient?  (Read 944 times)

Hobbit1972

  • Newbie
  • Posts: 4
how to speed up fphttpclient?
« on: June 04, 2021, 07:54:34 am »
Hi,

I need to retrieve several https documents. Using fphttpclient this is very slow:
Code: Pascal  [Select][+][-]
  1.   InitSSLInterface;
  2.   HTTPClient := tfphttpclient.create(nil);
  3.   with HTTPClient do begin
  4.     AllowRedirect := true;
  5.     AddHeader('accept','application/json');
  6.     AddHeader('Cache-Control','no-cache');
  7.   end;
  8.  
  9.   ...
  10.   response := HTTPClient.get(requeststr);  
  11.  

In perl I sped up things by using caching:
Code: Pascal  [Select][+][-]
  1. my $ua = LWP::UserAgent->new();
  2. $ua->protocols_allowed( [ 'http','https'] );
  3. $ua->ssl_opts( verify_hostnames => 0 );    
  4. my $conn_cache = LWP::ConnCache->new;
  5. $conn_cache->total_capacity([1]) ;
  6. $ua->conn_cache($conn_cache) ;
  7.  

Is there caching in fphttpclient too? Or do I need to switch to another package?

Thanks!

engkin

  • Hero Member
  • *****
  • Posts: 2857
Re: how to speed up fphttpclient?
« Reply #1 on: June 04, 2021, 04:43:12 pm »
Probably using KeepConnection should be enough:
Code: Pascal  [Select][+][-]
  1.   c.KeepConnection:=True;

But if you want to have more control, provide your own ssl handler. To disable the verification of the certificate for instance:
Code: Pascal  [Select][+][-]
  1. uses ... openssl, fphttpclient, opensslsockets, sslsockets;
  2.  
  3. type
  4.  
  5.   TExOpenSSLSocketHandler=class(TOpenSSLSocketHandler)
  6.   public
  7.     constructor Create;override;
  8.   end;
  9.  
  10. constructor TExOpenSSLSocketHandler.Create;
  11. begin
  12.   inherited Create;
  13.   VerifyPeerCert:=False;
  14. end;
  15.  

And set it as the default handler. Assuming you understand what that means:
Code: Pascal  [Select][+][-]
  1. var
  2.   c:TFPHTTPClient;
  3.  
  4. begin
  5.   InitSSLInterface;
  6.   TSSLSocketHandler.SetDefaultHandlerClass(TExOpenSSLSocketHandler);
  7.  
  8.   c:=TFPHTTPClient.Create(nil);
  9.   c.KeepConnection:=True;
  10.   ...

I did not test the above code. It is just to give you the idea.

Hobbit1972

  • Newbie
  • Posts: 4
Re: how to speed up fphttpclient?
« Reply #2 on: June 04, 2021, 05:31:19 pm »
Probably using KeepConnection should be enough:
Code: Pascal  [Select][+][-]
  1.   c.KeepConnection:=True;
Ah - thank you. That alone made execution about three times faster.


But if you want to have more control, provide your own ssl handler. ...
This did not give any more benefits - but might do once the overall number of http-gets increases.

Thx!l

engkin

  • Hero Member
  • *****
  • Posts: 2857
Re: how to speed up fphttpclient?
« Reply #3 on: June 04, 2021, 06:35:04 pm »
But if you want to have more control, provide your own ssl handler. ...
This did not give any more benefits - but might do once the overall number of http-gets increases.

Unless you deal with many servers, no benefit is expected from it when KeepConnection works because it runs *once* in this case.
« Last Edit: June 04, 2021, 06:46:41 pm by engkin »

engkin

  • Hero Member
  • *****
  • Posts: 2857
Re: how to speed up fphttpclient?
« Reply #4 on: June 04, 2021, 07:27:04 pm »
If you still need more speed, your next step would be using threads.

 

TinyPortal © 2005-2018