Lazarus

Programming => Operating Systems => Mac OS X => Topic started by: kevin.black on March 01, 2019, 06:38:25 am

Title: Simple DYLIB Test (Works Carbon, NOT Cocoa)
Post by: kevin.black on March 01, 2019, 06:38:25 am
Hi,

I cranked up a simple dylib test application after trying with no success to convert a slightly more complicated test app and dylib. I was using Trunk/Trunk, but have now gone back to 2.0.0RC3 FWIW. On the plus side, I think I turned on verbose logging on the 2.0.TC3 version and I suspect that is showing what's missing.

The simple application worked in i386/Carbon, but now does not in x86_64/Cocoa. It's the same test code I mentioned in this post: http://forum.lazarus-ide.org/index.php?topic=43925.msg308247#msg308247 (http://forum.lazarus-ide.org/index.php?topic=43925.msg308247#msg308247)  posted later, but the link error I'm getting when trying to build the simple dylib is cannot find:

Quote
    /usr/lib/dylib1.10.5.o

I have absolutely no idea what this file is and I can confirm I certainly don't have it. If I had to guess I'd say it ha something to do with macOS (but 10.5 was a millennia ago so probably not). I'm currently using 10.14.

And if I set {$mode delphi} Then this file cannot be found:

Quote
    /usr/lib/crt1.10.5.o

Does anyone have an idea of what dylib1.10.5.o is and how I might either get a copy of fix it so that the system doesn't need it?

The code is below (and it did work with i386/carbon). If it's TL;DR then fine, I just need some help with dylib1.10.5.o and/or crt1.10.5.o?

The DYLIB:

Code: Text  [Select]
  1.  This is the new direct call: DYLIB / DLL using FMX.Forms and FMX.Dialogs');
  2.  
  3.   {$IFDEF WINDOWS}
  4.   sType := 'Windows DLL';
  5.   {$ENDIF WINDOWS}
  6.   {$IFDEF DARWIN}
  7.   sType := 'macOS DYLIB';
  8.   {$ENDIF DARWIN}
  9.   Result := True;
  10. end;
  11.  
  12.  
  13. end.
  14.  
  15. exports
  16.   //say_Hello;{$IFDEF DARWIN} name '_say_Hello'; {$ENDIF DARWIN}
  17.   say_Hello;
  18.  
  19. initialization
  20.   ShowMessage('[DYLIB] initialization');
  21.  
  22. finalization
  23.   ShowMessage('[DYLIB] finalization');
  24.  
  25. end.

The Test App:
Code: Pascal  [Select]
  1. unit uDYLIBTestAPP;
  2.  
  3. {$mode objfpc}{$H+}
  4.  
  5. interface
  6.  
  7. uses
  8.   Classes,
  9.   SysUtils,
  10.   Forms,
  11.   Controls,
  12.   Graphics,
  13.   Dialogs,
  14.   Buttons,
  15.   StdCtrls;
  16.  
  17. type
  18.  
  19.   { TfDylibTestApp }
  20.  
  21.   TfDylibTestApp = class(TForm)
  22.     BitBtn1: TBitBtn;
  23.     BitBtn2: TBitBtn;
  24.     BitBtn3: TBitBtn;
  25.     Memo1: TMemo;
  26.     procedure BitBtn1Click(Sender: TObject);
  27.     procedure BitBtn2Click(Sender: TObject);
  28.     procedure BitBtn3Click(Sender: TObject);
  29.   private
  30.  
  31.   public
  32.  
  33.   end;
  34.  
  35. const
  36.   // Windows DLL Names
  37.   {$IFDEF MSWINDOWS}
  38.   TestDLL = 'pdylibtest.dll';
  39.   {$ENDIF MSWINDOWS}
  40.  
  41.   // macOS DYLIB Names
  42.   {$IFDEF DARWIN}
  43.   TestDLL = 'pdylibtest';
  44.   {$ENDIF DARWIN}
  45.  
  46.   {$IFDEF MSWINDOWS}
  47.     function say_Hello(Hello: string): boolean; stdcall; external TestDLL Delayed;
  48.   {$ENDIF MSWINDOWS}
  49.  
  50.   {$IFDEF DARWIN}
  51.    function say_Hello(Hello: string): boolean; cdecl; external TestDLL;
  52.   {$ENDIF DARWIN}
  53.  
  54.   function localFunction(localIn: string; out localOut: string): boolean;
  55.  
  56. var
  57.   fDylibTestApp: TfDylibTestApp;
  58.  
  59. implementation
  60.  
  61. function localFunction(localIn: string; out localOut: string): boolean;
  62. begin
  63.   LocalOut := 'This was passed to the function: ' + LocalIn;
  64.   Result := True;
  65. end;
  66.  
  67. {$R *.lfm}
  68.  
  69. { TfDylibTestApp }
  70.  
  71. procedure TfDylibTestApp.BitBtn3Click(Sender: TObject);
  72. begin
  73.   // Quit the test application
  74.   close;
  75. end;
  76.  
  77. procedure TfDylibTestApp.BitBtn2Click(Sender: TObject);
  78. var
  79.   sOut: string;
  80.  
  81. begin
  82.   // This is a local function
  83.   ShowMessage('Local Function Call');
  84.   if localFunction('Local in', sOut) then
  85.     showmessage('Local Function: TRUE (' + sOut + ')')
  86.   else
  87.     showmessage('Local Function: FALSE');
  88. end;
  89.  
  90. procedure TfDylibTestApp.BitBtn1Click(Sender: TObject);
  91.  
  92. var
  93.    b:boolean;
  94.    sType: string;
  95.    sDLLString: string;
  96.  
  97. begin
  98.  
  99.     b := False;
  100.  
  101.     // Call the DLL Function
  102.     {$IFDEF MSWINDOWS}
  103.     sType := 'Windows DLL';
  104.     sDLLString := 'The string passed to the Windows DLL';
  105.     b := say_Hello(sDLLString);
  106.     {$ENDIF MSWINDOWS}
  107.     {$IFDEF DARWIN}
  108.     sType := 'macOS DYLIB';
  109.     sDLLString := 'The string passed to the macOS DYLIB';
  110.     b := say_Hello(sDLLString);
  111.     {$ENDIF DARWIN}
  112.  
  113.     if b then
  114.       showmessage('Returned From: ' + sType + ': TRUE')
  115.     else
  116.       showmessage('Returned From: ' + sType + ': FALSE');
  117.  
  118. end;
  119.  
  120. end.
  121.  
Title: Re: Simple DYLIB Test (Works Carbon, NOT Cocoa)
Post by: kevin.black on March 01, 2019, 07:13:57 am
So after some digging I found this: http://forum.lazarus.freepascal.org/index.php?topic=42657.0 (http://forum.lazarus.freepascal.org/index.php?topic=42657.0)
Code: Pascal  [Select]
  1. sudo ln -s /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/lib/crt1.10.5.o  /usr/lib/crt1.10.5.o

But, of course that would be too simple, ket getting a ...you cannot do that.. error so after more digging: https://apple.stackexchange.com/questions/208478/how-do-i-disable-system-integrity-protection-sip-aka-rootless-on-macos-os-x (https://apple.stackexchange.com/questions/208478/how-do-i-disable-system-integrity-protection-sip-aka-rootless-on-macos-os-x)

Apparently after 10.6 Apple included System Integrity Protection (SIP) or as I like to call it 'Why don't you f$%^&ing make it as difficult as possible to do anything'. Anyway, the issue with dylib1.10.5.o and crt1.10.5.o are now gone.

I am back to my original issue of I cannot build the test application successfully because it cannot find the function within the DYLIB. I'm sure this is an issue with one or more settings although I have tried use about every permutation.
Quote
Compile Project, Target: x86_64-darwin/pDYLIBTestAPP: Exit code 256, Errors: 3, Hints: 2
Hint: Start of reading config file /etc/fpc.cfg
Hint: End of reading config file /etc/fpc.cfg
Verbose: Free Pascal Compiler version 3.0.4 [2017/11/26] for x86_64
Verbose: Copyright (c) 1993-2017 by Florian Klaempfl and others
Verbose: Target OS: Darwin for x86_64
Verbose: Compiling pDYLIBTestAPP.lpr
Verbose: Assembling (pipe) /Users/kevin/Dropbox/Lazarus/DYLIBTest/lib/x86_64-darwin/pDYLIBTestAPP.s
Verbose: Compiling resource /Users/kevin/Dropbox/Lazarus/DYLIBTest/lib/x86_64-darwin/pDYLIBTestAPP.or
Verbose: Linking /Users/kevin/Dropbox/Lazarus/DYLIBTest/x86_64-darwin/pDYLIBTestAPP
Error: linker: Undefined symbols for architecture x86_64:
Error: linker:   "_sayHello", referenced from:
Debug:   "_sayHello", referenced from:
Debug:       _UDYLIBTESTAPP$_$TFDYLIBTESTAPP_$__$$_BITBTN1CLICK$TOBJECT in udylibtestapp.o
ld: symbol(s) not found for architecture x86_64
An error occurred while linking
pDYLIBTestAPP.lpr(24,0) Error: Error while linking
pDYLIBTestAPP.lpr(24,0) Verbose: There were 1 errors compiling module, stopping
Verbose: Compilation aborted

Can anyone provide me with a definitive list of settings (either Lazarus OPTIONS or application OPTIONS) to allow a test application to call a function in a DYLIB. The DYLIB is statically linked so there is nothing fancy about trying to dynamically link and unlink the DYLIB.

Thanks,
Kevin
Title: Re: Simple DYLIB Test (Works Carbon, NOT Cocoa)
Post by: Jonas Maebe on March 01, 2019, 08:36:02 am
Normally, if you installed FPC 3.0.4a (from https://sourceforge.net/projects/freepascal/files/Mac%20OS%20X/3.0.4 ) after you installed macOS 10.14, and if you also (re)installed the command line tools after upgrading to macOS 10.14 (with sudo xcode-select --install), the FPC installer will have created a /etc/fpc.cfg file that contains the appropriate parameters so that this file can be found.

However, from another thread where someone used FPCUpDeluxe, it seems that program creates an alternate configuration file that overrides these settings. See http://forum.lazarus-ide.org/index.php/topic,44453.msg312787.html#msg312787 for the command line parameters to add so that the compiler/linker can find these files in case the fpc.cfg that contains them is not used for some reason.
Title: Re: Simple DYLIB Test (Works Carbon, NOT Cocoa)
Post by: kevin.black on March 02, 2019, 01:24:24 am
@Jonas Maebe

Thank you for the explanation, it is quite confusing (not your explanation, but the way FPC and FPCUPDeluxe does thins). I have referenced your post in the FPCUpDeluxe thread, hopefully it will assist others.

WRT getting the app/dylib working, here's what I have done so far and I now have the app referencing the function in the DYLIB(NOTE: Switched to i396/Carbon):
I will now simply substitute the compilers and widget sets and see if both build under x86_64/Cocoa.

FWIW here is the working code and, yes, I know it's not rocket science, but this actually took a long time:

DYLIB:
Code: Pascal  [Select]
  1. library pDYLIBTest;
  2.  
  3. {$mode objfpc}{$H+}
  4.  
  5. {$R *.res}
  6.  
  7. uses
  8.   Classes,
  9.   SysUtils,
  10.   Forms,
  11.   Controls,
  12.   Graphics,
  13.   StdCtrls,
  14.   Dialogs,
  15.   Buttons,
  16.   Interfaces;
  17.  
  18. var
  19.   sType: string;
  20.  
  21. {$IFDEF WINDOWS}
  22. function sayHello(Hello: string): boolean; stdcall; export;
  23. {$ENDIF WINDOWS}
  24. {$IFDEF DARWIN}
  25. function sayHello(Hello: string): boolean; cdecl; export;
  26. {$ENDIF DARWIN}
  27. begin
  28.   ShowMessage('[DYLIB] This is the new direct call: DYLIB / DLL using FMX.Forms and FMX.Dialogs');
  29.  
  30.   {$IFDEF WINDOWS}
  31.   sType := 'Windows DLL';
  32.   {$ENDIF WINDOWS}
  33.   {$IFDEF DARWIN}
  34.   sType := 'macOS DYLIB';
  35.   {$ENDIF DARWIN}
  36.   Result := True;
  37. end;
  38.  
  39. exports
  40.   sayHello {$IFDEF DARWIN} name '_sayHello'; {$ENDIF DARWIN}
  41.  
  42. initialization
  43.   ShowMessage('[DYLIB] initialization');
  44.  
  45. finalization
  46.   ShowMessage('[DYLIB] finalization');
  47.  
  48. end.

Test App:
Code: Pascal  [Select]
  1. unit uDYLIBTestAPP;
  2.  
  3. {$mode objfpc}{$H+}
  4.  
  5. interface
  6.  
  7. uses
  8.   Classes,
  9.   SysUtils,
  10.   Buttons,
  11.   Controls,
  12.   Stdctrls,
  13.   Dialogs,
  14.   Forms,
  15.   Menus;
  16.  
  17. type
  18.  
  19.   { TfDylibTestApp }
  20.  
  21.   TfDylibTestApp = class(TForm)
  22.     BitBtn1: TBitBtn;
  23.     BitBtn2: TBitBtn;
  24.     BitBtn3: TBitBtn;
  25.     MainMenu1: TMainMenu;
  26.     Memo1: TMemo;
  27.     Preferences: TMenuItem;
  28.     MenuItem2: TMenuItem;
  29.     procedure BitBtn1Click(Sender: TObject);
  30.     procedure BitBtn2Click(Sender: TObject);
  31.     procedure BitBtn3Click(Sender: TObject);
  32.     procedure FormCreate(Sender: TObject);
  33.     procedure FormDestroy(Sender: TObject);
  34.   private
  35.  
  36.   public
  37.  
  38.   end;
  39. (* {$ifdef mswindows}{$ifdef wince}{$calling cdecl}{$else}{$calling stdcall}{$endif}{$elsif darwin}{$calling cdecl}{$else}{$calling default}{$endif}*)
  40. const
  41.   {$IFDEF MSWINDOWS}
  42.   TestDLL = 'pdylibtest.dll';
  43.   preName = '';
  44.   {$ENDIF MSWINDOWS}
  45.   {$IFDEF DARWIN}
  46.   TestDLL = 'libpdylibtest.dylib';
  47.   {$linklib libpdylibtest.dylib}
  48.   preName = '_';
  49.   {$ENDIF DARWIN}
  50.  
  51.   // External function (DYLIB)
  52.   {$IFDEF MSWINDOWS}
  53.   function sayHello(Hello: string): boolean; stdcall; external TestDLL;
  54.   {$ENDIF MSWINDOWS}
  55.   {$IFDEF DARWIN}
  56.   function sayHello(Hello: string): boolean; cdecl; external TestDLL;
  57.   {$ENDIF DARWIN}
  58.   // Internal function
  59.   function localFunction(localIn: string; out localOut: string): boolean;
  60.  
  61. var
  62.   fDylibTestApp: TfDylibTestApp;
  63.  
  64. implementation
  65.  
  66. function localFunction(localIn: string; out localOut: string): boolean;
  67. begin
  68.   LocalOut := 'This was passed to the function: ' + LocalIn;
  69.   Result := True;
  70.  
  71. end;
  72.  
  73. {$R *.lfm}
  74.  
  75. { TfDylibTestApp }
  76.  
  77. procedure TfDylibTestApp.BitBtn3Click(Sender: TObject);
  78. begin
  79.   // Quit the test application
  80.   showmessage('[DYLIBTestApp] Closing application');
  81.   fDylibTestApp.close;
  82.   Application.Terminate;
  83.  
  84. end;
  85. procedure TfDylibTestApp.FormCreate(Sender: TObject);
  86. begin
  87.   showmessage('[DYLIBTestApp] FormCreate');
  88.  
  89. end;
  90. procedure TfDylibTestApp.FormDestroy(Sender: TObject);
  91. begin
  92.    showmessage('[DYLIBTestApp] FormDestroy');
  93.  
  94. end;
  95. procedure TfDylibTestApp.BitBtn2Click(Sender: TObject);
  96. var
  97.   sOut: string;
  98.  
  99. begin
  100.   // This is a local function
  101.   ShowMessage('[DYLIBTestApp] Local Function Call');
  102.   if localFunction('Local in', sOut) then
  103.     showmessage('[DYLIBTestApp] Local Function: TRUE (' + sOut + ')')
  104.   else
  105.     showmessage('[DYLIBTestApp] Local Function: FALSE');
  106.  
  107. end;
  108. procedure TfDylibTestApp.BitBtn1Click(Sender: TObject);
  109. var
  110.    b:boolean;
  111.    sType: string;
  112.    sDLLString: string;
  113.  
  114. begin
  115.     b := False;
  116.  
  117.     // Call the DLL Function
  118.     {$IFDEF MSWINDOWS}
  119.     sType := 'Windows DLL';
  120.     sDLLString := 'The string passed to the Windows DLL';
  121.     {$ENDIF MSWINDOWS}
  122.     {$IFDEF DARWIN}
  123.     sType := 'macOS DYLIB';
  124.     sDLLString := 'The string passed to the macOS DYLIB';
  125.     {$ENDIF DARWIN}
  126.  
  127.     b := sayHello(sDLLString);
  128.     if b then
  129.       showmessage('[DYLIBTestApp] Returned From: ' + sType + ': TRUE')
  130.     else
  131.       showmessage('[DYLIBTestApp] Returned From: ' + sType + ': FALSE');
  132.  
  133. end;
  134.  
  135. end.
Title: Re: Simple DYLIB Test (Works Carbon, NOT Cocoa)
Post by: kevin.black on March 02, 2019, 02:17:26 am
And some anomalies for those that are interested:
The link error (which I do not get with 2.0.0RC3??????:
Quote
Compile Project, Target: x86_64-darwin/pDYLIBTestAPP: Exit code 1, Errors: 2
Error: ld: library not found for -lpdylibtest
An error occurred while linking
Error: Error while linking

With regard to the CLOSE not actually doing anything (and neither does Application.terminate). I was told in the Delphi world not to use Application.terminate and Halt because CLOSE actually performs a lot of functions like garbage collect etc etc. Just doing a brute force HALT will leave a lot of the cleanup undone. Is this the case with lazarus?

FWIW, CLOSE - Does not work, APPLICATION.TERMINATE - Does not Work, HALT - Does Work (good, but not sure if it's good practice).

So my plan at the moment is to use 2.0.0RC3 if for no other reason than it works?

Thanks,
Kevin
Title: Re: Simple DYLIB Test (Works Carbon, NOT Cocoa)
Post by: Jonas Maebe on March 02, 2019, 10:51:37 am
I'm not familiar with LCL/GUI programming, but in general on macOS closing the last window/form of an application should never terminate it (unless it's kind of a widget, like a calculator). Applications should only quit if the user explicitly quits it. That may be why the Close does not terminate the application. I would expect Application.Terminate to quit it though, but as mentioned, I have absolutely zero knowledge of the internals of the LCL. It's probably better to ask about this in a general LCL forum.

The export name issue is indeed annoying. It's a historically grown issue and probably cannot be fixed by breaking other things, which is why it has lingered for so long.

Regarding the link error you got with fpcupdeluxe and the latest version: did the library itself compile and link successfully?
Title: Re: Simple DYLIB Test (Works Carbon, NOT Cocoa)
Post by: kevin.black on March 04, 2019, 02:37:35 am
@Jonas Maebe

Regarding the link error you got with fpcupdeluxe and the latest version: did the library itself compile and link successfully?

Yes, it did.

Progress today: I have now got the TestApp and the DYLIB to communicate. Specifically the parameter is passed to the DYLIB and (the correct) data returned. I took this as a major breakthrough.

So I retrofitted all of the settings etc to another test app and a DLL I am converting from Delphi/Windows. I commented out all of the functions in that DLL except 3 and even commented out the contents of the 3 (JUST so I could prove that the test app actually got to the DYLIB). As noted, I retrofitted EVERY setting I could find. The Dylib built as expected.

Well, of course, the Test App cannot find any of the exported functions from the DYLIB. I have been forensically comparing settings to no avail. It is (extremely) frustrating and obviously to do with some sort of name mangling, but, like I said, I have forensically copied the setting of a working test app/dylib.

EDIT: Missed this compiler directive (Blinding Flash of the obvious)
Code: Pascal  [Select]
  1.   {$IFDEF DARWIN}
  2.   EmpyreanAPIDLL = 'libdbxserverapi.dylib';
  3.   {$linklib libdbxserverapi.dylib}   <<<<<<--------------
  4.   {$ENDIF DARWIN}

Regardless, thank you for the advice.

Kevin