Including OpenSSL in a cross-platform application

[maurobio]

Dear ALL,

Hi!

So, I have a cross-platform application which uses OpenSSL. When creating a version for Windows, it is simple enough to include the shared libraries "ssleay32.dll" and "libeay32.dll" in the project folder and that is that. But things become a little bit more complicated when creating a Linux version.

In Linux, the equivalent of the above shared libraries are "libcrypto.so" and "libssl.so", but to begin with, I have not been able to locate these files. Second, I downloaded static versions of these libraries from here https://indy.fulgan.com/SSL/, but I could not figure out how to compile the application with them included, which does not seem to be a very bright idea anyway. Ideally, I would like to use the dynamic versions of the libraries and put them in an adequate folder (eg. /usr/lib) when creating a distribution paclage.

Could someone out there please give me some hints on how to get this done?

Thanks in advace for any assistance you can provide.

With warmeest regards,

[delphius]

Could someone out there please give me some hints on how to get this done?

Linux usually has openssl libs pre-installed, but the latest version is .3, if you use the fpc trunk version of the module openssl.pas, then it already includes support for these versions:

Code: Pascal  [Select][+][-]

1. DLLVersions: array[1..20] of AnsiString = ('','.3', '.1.1', '.11', '.10', '.1.0.6', '.1.0.5', '.1.0.4', '.1.0.3',
2.                                         '.1.0.2', '.1.0.1','.1.0.0','.0.9.8',
3.                                         '.0.9.7', '.0.9.6', '.0.9.5', '.0.9.4',
4.                                         '.0.9.3', '.0.9.2', '.0.9.1');
5.  

If it is NOT, then you can use a trick:

Code: Pascal  [Select][+][-]

1.         if not InitSSLInterface then
2.           begin
3.             openssl.DLLVersions[1] := '.3';
4.           end;
5.         if not InitSSLInterface then
6.           begin
7.             exit; // no ssl
8.           end;

In my simple Indy fpmailsend repo there are latest .so biraries precompiled for linux.

Cause I use Indy so can use .so libs from program folder just with:

Code: Pascal  [Select][+][-]

1. ...
2. uses IdSSLOpenSSLHeaders;
3. ...
4. IdOpenSSLSetLibPath(ExtractFilePath(ParamStr(0)));
5. ...

[maurobio]

Ave, @delphius!

Gratias!

Cause I use Indy and so can use .so libs from program folder just with:

Aha! That's is the trick I was looking for! I will try it ASAP.

Thank you very much!

With warmest regards,

[maurobio]

Hi, @delphius!

I have found a problem with the Indy-based solution. It is not an error, but just a "quirk". As it happens, my application does not uses Indy, but the regular OpenSSL library included with Lazarus. Is there any way of getting a similar solution to that you offered, but one not one based on Indy?

Thank you very much!

With warmest regards,

[delphius]

Is there any way of getting a similar solution to that you offered, but one not one based on Indy?

May be you can use this?

[abouchez]

With our https://github.com/synopse/mORMot2/blob/master/src/lib/mormot.lib.openssl11.pas unit, we had to manage the changing names and detect the version to adapt the calls to some OpenSSL APIs at runtime.
Note that some OpenSSL APIs are inconsistent between 1.x and 3.x branches.
Check how our unit knows how to adapt: https://github.com/synopse/mORMot2/blob/master/src/lib/mormot.lib.openssl11.pas#L3360

This unit is not based on Indy or FPC RTL, and also works on Delphi.
It has some object-based wrappers to the most used structures, so it may be easier to work than plain OpenSSL calls.
And it is used on production on several systems (including Mac M1).

[Remy Lebeau]

In my simple Indy fpmailsend repo there are latest .so biraries precompiled for linux.

Cause I use Indy so can use .so libs from program folder just with:

Code: Pascal  [Select][+][-]

1. ...
2. uses IdSSLOpenSSLHeaders;
3. ...
4. IdOpenSSLSetLibPath(ExtractFilePath(ParamStr(0)));
5. ...

Just be aware that Indy dynamically loads the unversioned libs by default, which is becoming more of a problem as OpenSSL 1.1/3.0 distributions are becoming more common.  It means the libs are mapping to newer OpenSSL versions that are not compatible with Indy, causing runtime errors.  Fortunately, you can call IdOpenSSLSetCanLoadSymLinks(False) and/or IdOpenSSLSetLoadSymLinksFirst(False) at app startup to make Indy load the versioned libs instead, and then it will only load compatible versions.

[maurobio]

Hi, gentlemen!

I also found this possible simple solution:

https://forum.lazarus.freepascal.org/index.php?topic=21972.0

More soon...

Best regards,

[maurobio]

Gentlemen,

As it turned out, my problems are uglier than I thought at first. I will provide a concise description of them below, in the hope that some of you can enlighten me towards a solution.

I have a (relatively) simple cross-platform application that fetches data from several online databases to present information on biological species in an integrated format (the full source code can be found here: https://github.com/maurobio/especies.

It compiles and runs without problems under Windows and Linux but (there is always a "but"!) only in a machine that has Lazarus installed. On a machine which has not Lazarus installed, it fails with an "Access violation" error and that is that.

The application uses units fphttpclient, openssl, opensslsockets, DOM, XMLRead, and XPath; therefore it depends upon the OpenSSL and Expat libraries which are, of course, installed along with Lazarus in a regular installation of the IDE. I thought at first that it should be a simple matter of including these libraries in the application folder (at least under Windows), but this do not work. In Windows, I first put the required libraries (libeay32.dll, ssleay32.dll, libexpat.dll) in the application folder, but got an error saying something about the libraries not being compiled for use under Windows. I then installed the OpenSSL binaries (from here: https://slproweb.com/products/Win32OpenSSL.html), but it did not work either (that is, the application keeps issuing the "Access violation" error).

Under Linux the situation is, of course, a little bit more complicated. In an installation of Linux Mint 21 (without Lazarus installed), I found libssl installed but NOT libcrypto.

What I want it to provide these libraries to users, along with the installation packages for both the Windows and Linux versions. Under Linux, I create a Debian package for distribution the application and so I presume I should include these libraries in the list of dependencies. Under Windows, I use InnoSetup to create executable installation packages and I suppose I should include the required libraries in it to be put in the Windows/System folder. I have successfully deployed other application developed with FPC/Lazarus using both methods, but none of these applications are dependent of these libraries.

Are there some simple and clean ways of getting out of this mess?

Thanks in advance for any assistance you can provide!

With warmest regards,

[rvk]

I thought at first that it should be a simple matter of including these libraries in the application folder (at least under Windows), but this do not work. In Windows, I first put the required libraries (libeay32.dll, ssleay32.dll, libexpat.dll) in the application folder, but got an error saying something about the libraries not being compiled for use under Windows.

"A error saying something". Lovely description.
If you want help with this ALWAYS note the EXACT error message.

BTW. Did you match the bitness of the dll's correctly. (i.e. your application 64 bit, dll 64 bit / app 32 bit, dll 32 bit)

I then installed the OpenSSL binaries (from here: https://slproweb.com/products/Win32OpenSSL.html), but it did not work either (that is, the application keeps issuing the "Access violation" error).

Under Linux the situation is, of course, a little bit more complicated. In an installation of Linux Mint 21 (without Lazarus installed), I found libssl installed but NOT libcrypto.

libcrypto is something else. Why do you need that?

Under Windows, I use InnoSetup to create executable installation packages and I suppose I should include the required libraries in it to be put in the Windows/System folder.

I would not recommend putting anything in the Windows/System folder.
You can really mess things up. What version are you going to install? 32 bit, 64 bit, and where? In System32 or in SysWOW64? And which version where?
No, don't do that.

If your program is 32 bit, just provide the 32 bit versions of the dll's in the executable directory.
And if it's 64 bit, provide the 64 bit versions.
Don't mix them.

On Linux it's best to include dependencies and let the correct libraries install (otherwise you end up in a dependency hell).

So, first things first, remove all the dll's from your exe directory (and if you installed openssl on Windows remove it there too so you begin with a clean slate).
Then determine what you need. Where does your program fail.
Copy the openssl 32 or 64 dlls to your exe directory (make sure you use the correct ones).
And see then what errors you get.

[toby]

why not save yourself a lot of headaches and just use synapse

http://www.ararat.cz/synapse/doku.php/download

i'm surprised rvk didn't suggst this to you

[maurobio]

Hi, @toby!

why not save yourself a lot of headaches and just use synapse

No, thanks! I do not have neither the time nor the inspiration to re-write my application to use Synapse (with which I am not familiar anyway).

 

i'm surprised rvk didn't suggst this to you

That's may be because he is too busy scolding me and calling me dumb (as if I don't know the difference between a 32-bit and a 64-bit system, etc.).

Thank you!

[rvk]

That's may be because he is too busy scalding me and calling me dumb (as if I don't know the difference between a 32-bit and a 64-bit system, etc.).

I didn't call you dumb but that's the most common cause of your problem (besides you not defining exactly what the error message is).

But if you don't want my help... Also fine.

I also had some remarks/suggestions about your program but I think I'll keep them to myself now (if you think I was scolding you).

[maurobio]

FYI: one cannot and should presume that someone does not know obvious things (like the fact that a 32-bit dll requires a 32-application, which BTW is my case - I have a 32-bit application which claims to not recognize 32-bit dll's which are installed with FPC/Lazarus, if they are put in the application folder).

I have not much time nor patience for Windows (on which all sorts of stupid things can happen, except those you do expect), and would appreciate if someone provide me useful help with the Linux version, thank you.

[paweld]

Hi @maurobio,
@rvk asked a good question in my opinion, sometimes such obvious things (which are often overlooked) save a lot of time and nerves. e.g. I recently lost 2 days trying to fix a simple bug that wouldn't go away - the problem turned out to be that I was testing the old compilation (in the release version) all the time, and compiling the changes to the debug version - habit and fatigue lead to such stupid mistakes.

My knowledge of Linux is not very good, but I know that it places great emphasis on security, so for a long time most distributions (as not all) use openssl v3. That's why your application doesn't want to work - fphttpclient in fpc 3.2 doesn't support this version of the library.

So he recommends using one of the 3 suggestions you received:
- upgrade fpc to trunk - openssl 3 support has been added. you probably won't have to make any changes to your code due to this change, but keep in mind that this is a development version and may cause problems
- switching to synapse, which also supports openssl version 3. fphttpclient is very similar to synapse, so the change should not be so painful and labor-intensive.
- using mormot2, which also supports openssl 3/3.1. here you will have to make some changes in the code.

Well, and keep in mind that in 2 days support for openssl 1.1 ends - more about it wrote @abouchez: https://forum.lazarus.freepascal.org/index.php/topic,64539.msg490883.html#msg490883