Lazarus

Programming => Networking and Web Programming => Topic started by: Igor Kokarev on October 29, 2018, 01:44:12 pm

Title: How to use WinInet from Lazarus? Download a file and POST
Post by: Igor Kokarev on October 29, 2018, 01:44:12 pm
Hi,

Is it possible to use WinInet API from Lazarus to download a file from HTTPS server, or perform POST query (to implement online activation for my product).

I tried fphttpclient, but it requires OpenSSL dlls.
Title: Re: How to use WinInet from Lazarus? Download a file and POST
Post by: sstvmaster on October 29, 2018, 02:01:00 pm
You can download OpenSSL file here: https://indy.fulgan.com/SSL/ (openssl-1.0.2p-...)

Download version win32 or win64, unzip and copy libeayXX.dll and ssleayXX.dll in your program folder.
Title: Re: How to use WinInet from Lazarus? Download a file and POST
Post by: Igor Kokarev on October 29, 2018, 02:31:44 pm
Thanks,

Yes, it worked.

But is it possible to use WinInet in Lazarus?
Title: Re: How to use WinInet from Lazarus? Download a file and POST
Post by: Thaddy on October 29, 2018, 02:44:47 pm
Yes it is!... but only on Windows. (I believe it is also deprecated by microsoft, at least for serverside applications. See: https://msdn.microsoft.com/library/windows/desktop/aa384273  and https://docs.microsoft.com/en-us/windows/desktop/WinInet/enabling-internet-functionality )
Unless you have a reason, don't use it since it is not cross-platform. fcl-net / fcl-web and synapse and indy are all cross-platform.
Title: Re: How to use WinInet from Lazarus? Download a file and POST
Post by: Igor Kokarev on October 29, 2018, 02:55:28 pm
Thanks for you reply!

What is best and simpliest cross-platform API (Win + macOS) just to download a file, or perform POST request to a server?
Title: Re: How to use WinInet from Lazarus? Download a file and POST
Post by: fred on October 29, 2018, 03:09:36 pm
Simpliest would be SimpleGet:
http://wiki.lazarus.freepascal.org/fphttpclient (http://wiki.lazarus.freepascal.org/fphttpclient)
Title: Re: How to use WinInet from Lazarus? Download a file and POST
Post by: Thaddy on October 29, 2018, 04:11:42 pm
Simpliest would be SimpleGet:
http://wiki.lazarus.freepascal.org/fphttpclient (http://wiki.lazarus.freepascal.org/fphttpclient)
Yes, maybe, but atm. the synapse solution is still simpler since simpleget does not allow for redirects (yet). So you can get the dreaded 403...
As I wrote before, this one allows https, redirects and works almost always;
Code: Pascal  [Select][+][-]
  1. {$ifdef fpc}{$mode delphi}{$H+}{$endif}
  2. uses classes, httpsend, ssl_openssl;
  3. var
  4.   l:Tstrings;
  5. begin
  6.   try
  7.     l:=Tstringlist.create;
  8.     if HttpGetText('https://www.google.com',L) then  // this has a redirection as well as ssl enabled.
  9.       writeln(l.text);
  10.   finally
  11.     l.free;
  12.   end;
  13. end.

That said, the same can be done will a slight modification of the SimpleGet code. I'll see if I can submit a patch.

In the above code you may want to use httpgetbinary to download a file, btw. And use a stream. This was just for illustration.
Code: Pascal  [Select][+][-]
  1. {$ifdef fpc}{$mode delphi}{$H+}{$endif}
  2. uses classes, httpsend, ssl_openssl;
  3. var
  4.   S:TStream;
  5.   l:Tstrings;
  6. begin
  7.     S:=TFilestream.Create('testme.bin', fmCreate or fmOpenWrite);
  8.     L:=TStringlist.create;
  9.     try
  10.     if HttpGetBinary('https://www.google.com',S) then
  11.     begin
  12.       S.Free;
  13.       L.LoadFromFile('testme.bin');
  14.       writeln(l.text);
  15.     end;
  16.   finally
  17.     l.free;
  18.   end;
  19. end.

Note that I mixed the two but the essential part is how to download in binary format. (Q & D) But cross-platform.




Title: Re: How to use WinInet from Lazarus? Download a file and POST
Post by: Igor Kokarev on October 30, 2018, 12:16:05 pm
Thanks for your recommendations! I will check it.
Title: Re: How to use WinInet from Lazarus? Download a file and POST
Post by: Hansaplast on November 27, 2019, 01:51:25 pm
A little late, but I was looking for an "easy" way to download a file from a webserver using HTTPS and ran into this topic.


Under more recent macOS versions (at least as of Mojave), fphttpclient will not work.
Apple does not seem to like the OpenSSL library (default macOS setup!):


Code: Pascal  [Select][+][-]
  1. Invalid dylib load. Clients should not load the unversioned libcrypto dylib as it does not have a stable ABI.


As an alternative for macOS users, use the unit "ns_url_request" by Phil Hess (can be found here (https://macpgmr.github.io/ObjP/nsunits-src.zip) - Phil's Mac related page (https://macpgmr.github.io/)).
This does not require any extra libraries (beyond what comes with macOS).


A quick (and sloppy) function pulling in HTTPS content as a string:


Code: Pascal  [Select][+][-]
  1. uses ... ns_url_request ... // Note: "ns_url_request" uses Phil's "NSHelpers" unit.
  2.  
  3.  
  4. ...
  5.  
  6.  
  7. function TForm1.GetURLContent(aURL:string):string;
  8. var
  9.   HTTP: TNSHTTPSendAndReceive;
  10. begin
  11.   HTTP := TNSHTTPSendAndReceive.Create;
  12.   HTTP.Method   := 'GET';
  13.   HTTP.Address  := aURL;
  14.   HTTP.SendAndReceive(Result);
  15.   HTTP.Free;
  16. end;  
Title: Re: How to use WinInet from Lazarus? Download a file and POST
Post by: trev on December 04, 2019, 01:31:23 pm
Under more recent macOS versions (at least as of Mojave), fphttpclient will not work.

Works for me in macOS (Mojave, Catalina), FreeBSD and Linux. Note: I'm using trunk (see below).
Title: Re: How to use WinInet from Lazarus? Download a file and POST
Post by: trev on December 04, 2019, 01:33:54 pm
Is it possible to use WinInet API from Lazarus to download a file from HTTPS server, or perform POST query (to implement online activation for my product).

Yes! See below:

Code: Pascal  [Select][+][-]
  1. {$IFDEF WINDOWS}
  2. // Need to use Windows WinInet to avoid issue with HTTPS
  3. // needing two OpenSSL DLLs to be provided with application
  4. // if using TFPHttpClient.
  5. // The WinINet API also gets any connection and proxy settings
  6. // set by Internet Explorer. Blessing or curse?
  7.  
  8. function GetWebPage(const Url: string): string;
  9. var
  10.   NetHandle: HINTERNET;
  11.   UrlHandle: HINTERNET;
  12.   Buffer: array[0..1023] of Byte;
  13.   BytesRead: dWord;
  14.   StrBuffer: UTF8String;
  15. begin
  16.   Result := '';
  17.   BytesRead := Default(dWord);
  18.   NetHandle := InternetOpen('Mozilla/5.0(compatible; WinInet)', INTERNET_OPEN_TYPE_PRECONFIG, nil, nil, 0);
  19.  
  20.   // NetHandle valid?
  21.   if Assigned(NetHandle) then
  22.     Try
  23.       UrlHandle := InternetOpenUrl(NetHandle, PChar(Url), nil, 0, INTERNET_FLAG_RELOAD, 0);
  24.  
  25.       // UrlHandle valid?
  26.       if Assigned(UrlHandle) then
  27.         Try
  28.           repeat
  29.             InternetReadFile(UrlHandle, @Buffer, SizeOf(Buffer), BytesRead);
  30.             SetString(StrBuffer, PAnsiChar(@Buffer[0]), BytesRead);
  31.             Result := Result + StrBuffer;
  32.           until BytesRead = 0;
  33.         Finally
  34.           InternetCloseHandle(UrlHandle);
  35.         end
  36.       // o/w UrlHandle invalid
  37.       else
  38.         ShowMessage('Cannot open URL: ' + Url);
  39.     Finally
  40.       InternetCloseHandle(NetHandle);
  41.     end
  42.   // NetHandle invalid
  43.   else
  44.     raise Exception.Create('Unable to initialize WinInet');
  45. end;
  46. {$ENDIF}  
  47.  
Title: Re: How to use WinInet from Lazarus? Download a file and POST
Post by: Hansaplast on December 04, 2019, 01:45:48 pm
Hmm, interesting. My Mac's (Catalina) complain with the above error  :(  ...
Phil wrote a small document on sandboxing and macOS (link (https://macpgmr.github.io/ObjP/SandboxingAppsOnMac.html)) where he states as well;
Quote
OpenSSL on Mac has been deprecated for a long time, so you should be using one of Apple's APIs instead


And he refers to an Apple document (https://developer.apple.com/library/archive/documentation/Security/Conceptual/cryptoservices/SecureNetworkCommunicationAPIs/SecureNetworkCommunicationAPIs.html#//apple_ref/doc/uid/TP40011172-CH13-SW3) from June 2018, indicating OpenSSL is deprecated and its use is "strongly" discouraged:
Quote
... OpenSSL is deprecated in macOS and is not provided in iOS. Use of the Apple-provided OpenSSL libraries by apps is strongly discouraged.


Granted; Sandbox is not the same as non-Sandbox apps, but I did run into issues with my regular apps as well (not sandboxed, not in the Apple Store) and Phils code resolved it.


p.s. Thanks Trev for the WinInet code example - I'll give that a try as well (under Windows of course)  :)
Title: Re: How to use WinInet from Lazarus? Download a file and POST
Post by: trev on December 04, 2019, 01:56:10 pm
Hmm, interesting. My Mac's (Catalina) complain with the above error  :(

Here's the code I use without any issues on macOS, FreeBSD and Linux:

Code: Pascal  [Select][+][-]
  1. {$IFDEF UNIX}
  2. function GetWebPage(const aURL: string): string;
  3. var
  4.   Client: TFPHttpClient;
  5. begin
  6.   Client := TFPHttpClient.Create(nil);
  7.  
  8.   Try
  9.     Client.AllowRedirect := true;
  10.     Client.AddHeader('User-Agent', 'Mozilla/5.0(compatible; fpweb)');
  11.     Result := Client.Get(aURL);
  12.   except
  13.       on E: Exception do
  14.            ShowMessage('Retrieval of: ' + aURL + LineEnding
  15.                        + 'Failed with error: ' + E.Message + LineEnding
  16.                        + 'HTTP code: ' + IntToSTr(Client.ResponseStatusCode));
  17.   end;
  18. end;
  19. {$ENDIF}
  20.  

Quote
p.s. Thanks Trev for the WinInet code example - I'll give that a try as well (under Windows of course)  :)

Glad to help :)
Title: Re: How to use WinInet from Lazarus? Download a file and POST
Post by: MISV on December 05, 2019, 09:58:02 pm
Have you checked this thread.

https://community.idera.com/developer-tools/platforms/f/macos-platform/70310/issue-with-using-indy-on-catalina

It is two months old where Remy mentions having introduced a fix / new function call in Indy
Title: Re: How to use WinInet from Lazarus? Download a file and POST
Post by: BeniBela on December 10, 2019, 12:49:37 pm
My Internet Tools (http://benibela.de/sources_en.html#internettools) use wininet for downloading
TinyPortal © 2005-2018