Example for HTTPS client including certificate verification

[Doyenne]

Can anyone give me an example of how to download a web resource via HTTPS, including certificate verification? Preferably using the FCL (e.g. fcl-web), but Synapse is also OK.

I saw that in FCL trunk, there is a VerifySSLCertificate property in TFPHTTPClient, but then I need to write an event handler and get stuck.

Let's say that I would like to download a web page and https://badssl.com/ should work, but https://expired.badssl.com/ and https://wrong.host.badssl.com/ should be rejected.

This is what I have, but I don't know how to add certificate verification:

Code: Pascal  [Select][+][-]

1. program project1;
2.  
3. {$mode objfpc}{$H+}
4.  
5. uses SysUtils, fphttpclient, openssl, opensslsockets;
6.  
7. const URLS: array[0..2] of string = (
8.   'https://badssl.com/',            // should work
9.   'https://expired.badssl.com/',    // should be rejected
10.   'https://wrong.host.badssl.com/'  // should be rejected
11. );
12.  
13. procedure TryURLs;
14. var
15.   URL: string;
16. begin
17.   for URL in URLS do
18.     try
19.       with TFPHTTPClient.Create(nil) do
20.         try
21.           AllowRedirect := True;
22.           // do something to enable certificate verification??
23.           Get(URL);
24.         finally
25.           Free;
26.         end;
27.       WriteLn(URL, ' succeeded.');
28.     except
29.       on E: Exception do
30.         WriteLn(Format('%s failed! (%s)', [URL, E.Message]));
31.     end;
32. end;
33.  
34. begin
35.   TryURLs;
36. end.

Thanks for any hint in the right direction!

[PierceNg]

Can anyone give me an example of how to download a web resource via HTTPS, including certificate verification? Preferably using the FCL (e.g. fcl-web), but Synapse is also OK.

I saw that in FCL trunk, there is a VerifySSLCertificate property in TFPHTTPClient, but then I need to write an event handler and get stuck.

The example fcl-web/examples/httpclient/httpget.pas shows how to set up the event handler.

Edit: Doesn't look fully functional as this line prints empty string:

Code: [Select]

TEncoding.ASCII.GetAnsiString( aHandler.CertificateData.Certificate.Value)

[PierceNg]

libcurl has what you want. Below is libcurl/examples/testcurl.pp with my changes marked "<====":

Code: [Select]

{$mode objfpc}
{$H+}
program testcurl;

uses libcurl;

Var
  URL : Pchar = 'https://wrong.host.badssl.com'; // <==== Change URL
  hCurl : pCurl;

begin
  hCurl:= curl_easy_init;
  if Assigned(hCurl) then
    begin
    curl_easy_setopt(hCurl,CURLOPT_VERBOSE, [True]);
    curl_easy_setopt(hCurl,CURLOPT_URL,[URL]);
    curl_easy_setopt(hCurl, CURLOPT_SSL_VERIFYPEER, [True]); // <==== Add this line
    curl_easy_perform(hCurl);
    curl_easy_cleanup(hCurl);
    end;
end.
Running it:

Code: Text  [Select][+][-]

1. % ./testcurl
2. *   Trying 104.154.89.105:443...
3. * TCP_NODELAY set
4. * Connected to wrong.host.badssl.com (104.154.89.105) port 443 (#0)
5. * ALPN, offering h2
6. * ALPN, offering http/1.1
7. * successfully set certificate verify locations:
8. *   CAfile: /etc/ssl/certs/ca-certificates.crt
9.   CApath: /etc/ssl/certs
10. * SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
11. * ALPN, server accepted to use http/1.1
12. * Server certificate:
13. *  subject: CN=*.badssl.com
14. *  start date: Jan 22 16:33:26 2023 GMT
15. *  expire date: Apr 22 16:33:25 2023 GMT
16. *  subjectAltName does not match wrong.host.badssl.com
17. * SSL: no alternative certificate subject name matches target host name 'wrong.host.badssl.com'
18. * Closing connection 0

For expired:

Code: Text  [Select][+][-]

1. % ./testcurl
2. *   Trying 104.154.89.105:443...
3. * TCP_NODELAY set
4. * Connected to expired.badssl.com (104.154.89.105) port 443 (#0)
5. * ALPN, offering h2
6. * ALPN, offering http/1.1
7. * successfully set certificate verify locations:
8. *   CAfile: /etc/ssl/certs/ca-certificates.crt
9.   CApath: /etc/ssl/certs
10. * SSL certificate problem: certificate has expired
11. * Closing connection 0

[BeniBela]

My internet tools handle that.

Code: Pascal  [Select][+][-]

1. program Project1;
2.  
3. uses internetaccess, synapseinternetaccess;
4. var
5.   i: TInternetAccess;
6. begin
7.   i := TSynapseInternetAccess.create();
8.   i.get('https://badssl.com/');
9. //  i.get('https://expired.badssl.com/');
10. //  i.get('https://wrong.host.badssl.com/');
11. end.
12.  

I started with the Synapse HTTPS handling, but changed a lot

[domasz]

Does any of these solutions work with HTTPS without DLLs?

[PierceNg]

Does any of these solutions work with HTTPS without DLLs?

The DLLs provide the TLS functionality including crypto, certificate handling, wire protocols, etc. Not using those DLLs means using a pure Pascal implementation of TLS with equivalent functionality. AFAIK there is no such thing.

[Thaddy]

AFAIK there is no such thing.

Of course there is: our forum member Xor-el (the one with the unpronouncable name) took care of that.... (about 4 years ago). Only "problem" is you need to know what you are doing.

[Doyenne]

My internet tools handle that.

I'm highly impressed! I had to install the FLRE and POCO library as well before I could compile, but then it worked perfectly.

IMHO, this is how it should be in 2023: SSL verification easy, by default and out of the box. (IMHO using HTTPS without certificate checking does not make sense.)

However, your library is GPL (not LGPL) so unfortunately I cannot use it in my project.

[Doyenne]

If I try using Synapse trunk, same machine, same configuration, it doesn't work:

Code: Pascal  [Select][+][-]

1. program project1;
2.  
3. uses blcksock, ssl_openssl3;
4.  
5. const URL = 'badssl.com';
6.  
7. var
8.   Sock: TTCPBlockSocket;
9. begin
10.   Sock := TTCPBlockSocket.Create;
11.   try
12.     Sock.Connect(URL, '443');
13.     Sock.SSL.VerifyCert := True;
14.     Sock.SSLDoConnect;
15.     if Sock.LastError <>  0 then
16.     begin
17.       WriteLn('Error: ', Sock.LastErrorDesc);
18.       WriteLn('GetVerifyCert: ', Sock.SSL.GetVerifyCert);
19.     end
20.     else
21.       WriteLn('Success!');
22.   finally
23.     Sock.Free;
24.   end;
25. end.
26.  

Output:

Code: [Select]

Error: error:0A000086:SSL routines::certificate verify failed
GetVerifyCert: 20

The number 20 might mean that the root certificate could not be checked (I find OpenSSL messages always a bit unclear), because if I intentionally hide the root certificates to OpenSSL, I get verify error num=20:

Code: [Select]

$ openssl s_client -quiet -connect badssl.com:443 -CApath /tmp
depth=2 C = US, O = Internet Security Research Group, CN = ISRG Root X1
verify error:num=20:unable to get local issuer certificate
verify return:1
depth=1 C = US, O = Let's Encrypt, CN = R3
verify return:1
depth=0 CN = *.badssl.com
verify return:1

Adding

Code: [Select]

sock.SSL.CertCAFile := '/etc/ssl/certs/ISRG_Root_X1.pem'; does not solve the problem, still same result.

Any hints are welcome...

[PierceNg]

AFAIK there is no such thing.

Of course there is: our forum member Xor-el (the one with the unpronouncable name) took care of that.... (about 4 years ago). Only "problem" is you need to know what you are doing.

That's a crypto library, not a TLS implementation.

[Thaddy]

I am sorry, but all the code you need is there. If you do not understand it, ask.

[domasz]

I am sorry, but all the code you need is there. If you do not understand it, ask.

I don't. Care to explain?