Recent

Author Topic: Static Linking OpenSSL and OSX version  (Read 15066 times)

Grahame Grieve

  • Full Member
  • ***
  • Posts: 224
Static Linking OpenSSL and OSX version
« on: November 23, 2021, 10:30:27 am »
I am trying to link openSSL directly into my application to see if I can figure out how to distribute my application as a hardened run time. (Sure, I can put the two dylibs into my .app package and sign them, and make the whole thing a hardened run time, but libssl has a non-versioned dependency on libcrypto and OSX won't allow that in a hardened run time. I can set up a pipeline to do a custom compile of openSSL with a modified link dependency, according to the openSSL mailing list, but someone there suggested that I just link the openSSL code directly into the binary. Sounds like a good idea to me if it works, and $Linklib looks like it should do the job.

Well... not so easy. I link the .a into my code on a Mac M1 with the instruction:

{$IFDEF FPC}
  {$DEFINE STATICLOAD_OPENSSL}
  {$LINKLIB libcrypto.a}
  {$LINKLIB libssl.a}
{$ENDIF}
           
(The STATICLOAD_OPENSSL is a define in the openSSL 1.1x branch of Indy)

When I compile, I get this error:

Error: linker:   "___chkstk_darwin", referenced from:
Debug:       _BN_mod_exp_mont_consttime in libcrypto.a(bn_exp.o)
Debug:       _do_ssl3_write in libssl.a(rec_layer_s3.o)
Debug:       _tls_parse_ctos_cookie in libssl.a(extensions_srvr.o)
Debug:       _curve448_base_double_scalarmul_non_secret in libcrypto.a(curve448.o)
Debug:       _ERR_print_errors_cb in libcrypto.a(err_prn.o)
Error: ld: symbol(s) not found for architecture arm64

All the online help I can find suggests that this is OS version problem in the linker. And indeed, prior to that, I get 100s of errors like this:

Error: ld: warning: object file (/Users/grahame/work/tools/fpc/units/aarch64-darwin/rtl/sysinit.o) was built for newer macOS version (11.0) than being linked (10.8)

One of these for every pascal unit in my code, and one for every c module in the openSSL code, finishing like this:

Error: ld: warning: object file (/Users/grahame/work/openssl/libcrypto.a(p12_attr.o)) was built for newer macOS version (11.0) than being linked (10.8)

The environment make file is /usr/bin/make - is that the linker? How should I fix the macOS linking version error? and will that fix the openSSL linking error?

btw,
* if I define -Xt, then I get a different error: "Error: ld: library not found for -lc". Should I do something about this? c?
* if I define -WP11.0, which some posts here seem to suggest, I get that this is an unrecognised compiler option. (or any other number)

As you can probably tell, I'm completely ignorant about linking stuff, and I have no idea how to progress this...

btw, Lazarus + FPC = trunk yesterday. Mac = Mac M1 running Big Sur 11.6. Xcode v13.0. Are any other versions relevant?

Grahame Grieve

  • Full Member
  • ***
  • Posts: 224
Re: Static Linking OpenSSL and OSX version
« Reply #1 on: November 23, 2021, 10:39:00 am »
well, immediately after I posted this, I found https://wiki.freepascal.org/Mac_Installation_FAQ#ld:_library_not_found_for_-lc. Which is kind of helpful, except that it's not. Put these things on the command line - which command line? When I run Lazarus? I tried all the likely places in project options... not them...?

Alextp

  • Hero Member
  • *****
  • Posts: 1574
    • UVviewsoft
Re: Static Linking OpenSSL and OSX version
« Reply #2 on: November 23, 2021, 10:46:13 am »
I see that 'Project options' dialog gives the "Compiler Options / Custom Options" memo-field.

Grahame Grieve

  • Full Member
  • ***
  • Posts: 224
Re: Static Linking OpenSSL and OSX version
« Reply #3 on: November 23, 2021, 11:13:56 am »
Yep, it does. What would I put in there that might make a difference? because I haven't figured it out

Alextp

  • Hero Member
  • *****
  • Posts: 1574
    • UVviewsoft
Re: Static Linking OpenSSL and OSX version
« Reply #4 on: November 23, 2021, 11:35:00 am »
I am ignorant in this, but a guess: your .o object files must be recompiled on macOS 11, and now they are for 10.x.
« Last Edit: November 23, 2021, 11:36:39 am by Alextp »

trev

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 1817
  • Former Delphi 1-7, 10.2 user
Re: Static Linking OpenSSL and OSX version
« Reply #5 on: November 23, 2021, 11:43:37 am »
well, immediately after I posted this, I found https://wiki.freepascal.org/Mac_Installation_FAQ#ld:_library_not_found_for_-lc. Which is kind of helpful, except that it's not. Put these things on the command line - which command line? When I run Lazarus? I tried all the likely places in project options... not them...?

If you read what I wrote in that FAQ, you'll see it was in the context of compiling FPC on the command line with make. So, in a different context, you'll need to adapt the solution. I suggest adding "-XR/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk" to your /etc/ fpc.cfg file to solve any such issue in the future.

As to the OpenSSL issues, I would just avoid them altogether and use the Apple Network Framework which doesn't require additional libraries which is what I've been doing. See my articles in the Wiki on NSURLConnection (now deprecated) and NSURLSession (macOS 10.14 onwards).

If you wish to continue to pursue static linking OpenSSL libraries, you could refer to my article on macOS Static Libraries.

As to "object file xxx was built for newer macOS version (11.0) than being linked",  you need to set the compiler flag -WM (eg -WM10.x) to the version number of the SDK you want to build against. So, add -WM10.x (or whatever) to your /etc/fpc.cfg file.

Finally, how did you install FPC and Lazarus because these odd issues you're encountering are not normal and I've never encountered them using the official Lazarus pkg installers.
Lazarus 2.3, FPC 3.3.1 macOS 12.1 x86_64 Xcode 13.2
Lazarus 2.3, FPC 3.3.1 macOS 12.1 aarch64 Xcode 13.2
Lazarus 2.3, FPC 3.2.2 FreeBSD 13.0 amd64 VM
Lazarus 2.3, FPC 3.2.2 FreeBSD 12.2 amd64 VM
Lazarus 2.1 r61574 FPC 3.0.4 Ubuntu 20.04 VM
Lazarus 2.0.10 FPC 3.2.0 Win10 amd64 VM

Grahame Grieve

  • Full Member
  • ***
  • Posts: 224
Re: Static Linking OpenSSL and OSX version
« Reply #6 on: November 23, 2021, 11:43:52 am »
well, I just compiled them all afresh right now, either using the openssl build or compiling them on the M1 Mac I'm using to link them

Grahame Grieve

  • Full Member
  • ***
  • Posts: 224
Re: Static Linking OpenSSL and OSX version
« Reply #7 on: November 23, 2021, 11:57:27 am »
> If you read what I wrote in that FAQ, you'll see it was in the context of compiling FPC on the command line with make. > So, in a different context, you'll need to adapt the solution. I suggest adding "-XR/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk" to your /etc/ fpc.cfg file to solve any such issue in the future.

I don't have an /etc/fpc.cfg file? Should I create it?

> As to the OpenSSL issues, I would just avoid them altogether and use the Apple Network Framework which doesn't require additional libraries which is what I've been doing. See my articles in the Wiki on NSURLConnection (now deprecated) and NSURLSession (macOS 10.14 onwards).

I use openSSL extensively for way more than HTTP client - quite a bit of crypto stuff, and an HTTP server. And it's a cross platform app, so simply using NSURLConnection really doesn't seem like an option to me. (The apple people keep saying that to me too)

> If you wish to continue to pursue static linking OpenSSL libraries, you could refer to my article on macOS Static Libraries.

I'll read it again, and see if I'm ready to learn more from it.

> As to "object file xxx was built for newer macOS version (11.0) than being linked",  you need to set the compiler flag -WM (eg -WM10.8) to the version number of the SDK you want to build against. So, add -WM10.8 (or whatever) to your /etc/fpc.cfg file.

well, that fixed the linking problem with openSSL, and now I get lots of access violations to play with. Thanks.

> Finally, how did you install FPC and Lazarus because these odd issues you're encountering are not normal and I've never encountered them using the official Lazarus pkg installers.

Using fpclazup, this command: ./tools/fpclazup --noconfirm --fpcVersion=trunk.gitlab --lazVersion=trunk.gitlab --installdir=tools --include=anchordocking,lazprojectgroups,virtualtreeview,fpdebug

trev

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 1817
  • Former Delphi 1-7, 10.2 user
Re: Static Linking OpenSSL and OSX version
« Reply #8 on: November 23, 2021, 12:18:02 pm »
Quote from: trev
Finally, how did you install FPC and Lazarus because these odd issues you're encountering are not normal and I've never encountered them using the official Lazarus pkg installers.

Using fpclazup, this command: ./tools/fpclazup --noconfirm --fpcVersion=trunk.gitlab --lazVersion=trunk.gitlab --installdir=tools --include=anchordocking,lazprojectgroups,virtualtreeview,fpdebug

Aha. I think that goes some way to explaining some of the odd issues you have encountered.
Lazarus 2.3, FPC 3.3.1 macOS 12.1 x86_64 Xcode 13.2
Lazarus 2.3, FPC 3.3.1 macOS 12.1 aarch64 Xcode 13.2
Lazarus 2.3, FPC 3.2.2 FreeBSD 13.0 amd64 VM
Lazarus 2.3, FPC 3.2.2 FreeBSD 12.2 amd64 VM
Lazarus 2.1 r61574 FPC 3.0.4 Ubuntu 20.04 VM
Lazarus 2.0.10 FPC 3.2.0 Win10 amd64 VM

Grahame Grieve

  • Full Member
  • ***
  • Posts: 224
Re: Static Linking OpenSSL and OSX version
« Reply #9 on: November 23, 2021, 01:17:52 pm »
it's not clear how else to install Lazarus (trunk version), other than fpcLazup or FpcupDeluxe. Anyway, the access violations are because while the whole thing compiles, none of the openSSL code is actually linked in. I guess I need -Xt for that, which brings me back to ... should I have a /etc/fpc.cfg?
« Last Edit: November 23, 2021, 01:39:24 pm by Grahame Grieve »

DonAlfredo

  • Hero Member
  • *****
  • Posts: 1530
Re: Static Linking OpenSSL and OSX version
« Reply #10 on: November 23, 2021, 01:55:14 pm »
We solved the -lc error together on Github. But this was limited to the compiler build itself.
This issue seems to make it clear that the same method should be used for FPC itself during use.
That would indeed mean an addition into fpc.cfg.
Will have a look.

trev

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 1817
  • Former Delphi 1-7, 10.2 user
Re: Static Linking OpenSSL and OSX version
« Reply #11 on: November 23, 2021, 11:57:03 pm »
it's not clear how else to install Lazarus (trunk version), other than fpcLazup or FpcupDeluxe.

I have instructions on the Wiki (Installing Lazarus on macOS) for installing from source.

I also have daily automated snapshot builds of both aarch64 FPC and aarch64 Lazarus development (trunk in svn speak or main in git speak) versions available for download from  SourceForge.
Lazarus 2.3, FPC 3.3.1 macOS 12.1 x86_64 Xcode 13.2
Lazarus 2.3, FPC 3.3.1 macOS 12.1 aarch64 Xcode 13.2
Lazarus 2.3, FPC 3.2.2 FreeBSD 13.0 amd64 VM
Lazarus 2.3, FPC 3.2.2 FreeBSD 12.2 amd64 VM
Lazarus 2.1 r61574 FPC 3.0.4 Ubuntu 20.04 VM
Lazarus 2.0.10 FPC 3.2.0 Win10 amd64 VM

Remy Lebeau

  • Hero Member
  • *****
  • Posts: 1042
    • Lebeau Software
Re: Static Linking OpenSSL and OSX version
« Reply #12 on: November 27, 2021, 06:50:23 am »
{$IFDEF FPC}
  {$DEFINE STATICLOAD_OPENSSL}
  {$LINKLIB libcrypto.a}
  {$LINKLIB libssl.a}
{$ENDIF}
           
(The STATICLOAD_OPENSSL is a define in the openSSL 1.1x branch of Indy)

First, that branch is OLD, and probably needs to be deleted. Indy's latest OpenSSL 1.1x support code is in this pull request instead.

Second, Indy is coded to support static linking of OpenSSL only on iOS systems (see the IdSSLOpenSSLHeaders_static.pas unit). But even so, STATICLOAD_OPENSSL is not something you can just define outside of Indy and have it magically take effect inside of Indy. That define needs to be enabled inside of Indy's own code, specifically in the IdCompilerDefines.inc file(s), and then Indy would need to be recompiled.
« Last Edit: November 27, 2021, 06:52:02 am by Remy Lebeau »
Remy Lebeau
Lebeau Software - Owner, Developer
Internet Direct (Indy) - Admin, Developer (Support forum)

Grahame Grieve

  • Full Member
  • ***
  • Posts: 224
Re: Static Linking OpenSSL and OSX version
« Reply #13 on: November 28, 2021, 11:17:42 am »
@trev:

> I have instructions on the Wiki (Installing Lazarus on macOS) for installing from source.

I tried to follow them, but all the compiles on current source fail. Maybe I have to install older versions from the downloads first (that's not clear), before I can build the current versions, but the downloads don't support M1 Macs... I couldn't figure out from that page what I actually have to do to get things to work on my Mac M1

@Remy

I am using the openSSL 1.1 branch of Indy (well, my own custom hack of it for memory management and thread tracking reasons). So I have to figure out how to statically bind openSSL on OSX, and yes, of course, Indy will have to be recompiled for that

Remy Lebeau

  • Hero Member
  • *****
  • Posts: 1042
    • Lebeau Software
Re: Static Linking OpenSSL and OSX version
« Reply #14 on: November 29, 2021, 08:01:02 pm »
@Remy

I am using the openSSL 1.1 branch of Indy

Again, that is an OLD branch, you really shouldn't be using it at all.
Remy Lebeau
Lebeau Software - Owner, Developer
Internet Direct (Indy) - Admin, Developer (Support forum)

 

TinyPortal © 2005-2018