On windows you need to create the directory c:\temp\ by hand where openssl saves the certificates.
Why C:\Temp ??? Surely you can make OpenSSL create the certificates in another place...
Besides that... I don't see anything in TFPWebModule that has to do with openssl and the certificates.
So I think it's only meant for HTTP and doesn't do HTTPS.
And yes... directly querying via Lazarus will get you the page via HTTP (the port for HTTP doesn't matter... you can even do HTTP over port 443).
Real web-browsers however (depending on the browser) do expect a https handshake (on port 443) to do HTTPS and will give you the warning if it's not done.