Forum > Networking and Web Programming

how to speed up fphttpclient?

(1/1)

Hobbit1972:
Hi,

I need to retrieve several https documents. Using fphttpclient this is very slow:

--- Code: Pascal  [+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i < x.length; i++) { x[i].style.maxHeight='none'; x[i].style.height = Math.min(x[i].clientHeight+15,306)+'px'; x[i].style.resize = "vertical";}};} ---  InitSSLInterface;  HTTPClient := tfphttpclient.create(nil);  with HTTPClient do begin    AllowRedirect := true;    AddHeader('accept','application/json');    AddHeader('Cache-Control','no-cache');  end;    ...  response := HTTPClient.get(requeststr);    
In perl I sped up things by using caching:

--- Code: Pascal  [+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i < x.length; i++) { x[i].style.maxHeight='none'; x[i].style.height = Math.min(x[i].clientHeight+15,306)+'px'; x[i].style.resize = "vertical";}};} ---my $ua = LWP::UserAgent->new();$ua->protocols_allowed( [ 'http','https'] );$ua->ssl_opts( verify_hostnames => 0 );    my $conn_cache = LWP::ConnCache->new;$conn_cache->total_capacity([1]) ;$ua->conn_cache($conn_cache) ; 
Is there caching in fphttpclient too? Or do I need to switch to another package?

Thanks!

engkin:
Probably using KeepConnection should be enough:

--- Code: Pascal  [+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i < x.length; i++) { x[i].style.maxHeight='none'; x[i].style.height = Math.min(x[i].clientHeight+15,306)+'px'; x[i].style.resize = "vertical";}};} ---  c.KeepConnection:=True;
But if you want to have more control, provide your own ssl handler. To disable the verification of the certificate for instance:

--- Code: Pascal  [+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i < x.length; i++) { x[i].style.maxHeight='none'; x[i].style.height = Math.min(x[i].clientHeight+15,306)+'px'; x[i].style.resize = "vertical";}};} ---uses ... openssl, fphttpclient, opensslsockets, sslsockets; type   TExOpenSSLSocketHandler=class(TOpenSSLSocketHandler)  public    constructor Create;override;  end; constructor TExOpenSSLSocketHandler.Create;begin  inherited Create;  VerifyPeerCert:=False;end; 
And set it as the default handler. Assuming you understand what that means:

--- Code: Pascal  [+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i < x.length; i++) { x[i].style.maxHeight='none'; x[i].style.height = Math.min(x[i].clientHeight+15,306)+'px'; x[i].style.resize = "vertical";}};} ---var  c:TFPHTTPClient; begin  InitSSLInterface;  TSSLSocketHandler.SetDefaultHandlerClass(TExOpenSSLSocketHandler);   c:=TFPHTTPClient.Create(nil);  c.KeepConnection:=True;  ...
I did not test the above code. It is just to give you the idea.

Hobbit1972:

--- Quote from: engkin on June 04, 2021, 04:43:12 pm ---Probably using KeepConnection should be enough:

--- Code: Pascal  [+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i < x.length; i++) { x[i].style.maxHeight='none'; x[i].style.height = Math.min(x[i].clientHeight+15,306)+'px'; x[i].style.resize = "vertical";}};} ---  c.KeepConnection:=True;
--- End quote ---
Ah - thank you. That alone made execution about three times faster.



--- Quote from: engkin on June 04, 2021, 04:43:12 pm ---But if you want to have more control, provide your own ssl handler. ...

--- End quote ---
This did not give any more benefits - but might do once the overall number of http-gets increases.

Thx!l

engkin:

--- Quote from: Hobbit1972 on June 04, 2021, 05:31:19 pm ---
--- Quote from: engkin on June 04, 2021, 04:43:12 pm ---But if you want to have more control, provide your own ssl handler. ...

--- End quote ---
This did not give any more benefits - but might do once the overall number of http-gets increases.

--- End quote ---

Unless you deal with many servers, no benefit is expected from it when KeepConnection works because it runs *once* in this case.

engkin:
If you still need more speed, your next step would be using threads.

Navigation

[0] Message Index

Go to full version