Recent

Author Topic: Dynamically linked executables...possible?  (Read 3188 times)

TCH

  • Full Member
  • ***
  • Posts: 147
Dynamically linked executables...possible?
« on: October 12, 2021, 04:34:04 pm »
In C there is the option (-Wl,--export-dynamic or -rdynamic) to link a binary dynamically. I found no option for this in the manual and according to this topic, it was not possible 3 years ago. Is it possible now?

marcov

  • Administrator
  • Hero Member
  • *
  • Posts: 9589
  • FPC developer.
Re: Dynamically linked executables...possible?
« Reply #1 on: October 12, 2021, 07:39:33 pm »
The FPC equivalent of -Wl is -k. So  -k--export-dynamic should do the same.

The question of course is if the object files are compatible with that, but that needs digging into the linker and its options

TCH

  • Full Member
  • ***
  • Posts: 147
Re: Dynamically linked executables...possible?
« Reply #2 on: October 12, 2021, 10:12:15 pm »
Thank you. Unfortunately this did not work, but i might done it wrong. To be precise: i've made a simple test library, which is dynamically loaded by the test executable, but one of library's functions calls another one which resides in the executable and not the library. The statical approach (passing the address of that function to the library) works perfectly, but i am curious if the same dynamical approach can be applied in Pascal as in C.

The library code:
Code: Pascal  [Select][+][-]
  1. library mylib;
  2.  
  3. {$mode objfpc}{$H+}
  4.  
  5. type TCaller = function (ptr: pinteger): integer; cdecl;
  6.  
  7. var caller: TCaller; external name 'caller';
  8.  
  9. function mylib_a(a: integer): integer; cdecl;
  10. begin
  11.         result := a - 5;
  12. end;
  13.  
  14. procedure mylib_b(b: pinteger); cdecl;
  15. begin
  16.         b^ := b^ - 5;
  17. end;
  18.  
  19. function mylib_c(c: pinteger): integer; cdecl;
  20. begin
  21.         c^ := c^ - 5;
  22.         result := c^ - 5;
  23. end;
  24.  
  25. function mylib_d(d: pinteger): integer; cdecl;
  26. begin
  27.         d^ := d^ + 5;
  28.         result := caller(d);
  29. end;
  30.  
  31. exports mylib_a, mylib_b, mylib_c, mylib_d;
  32.  
  33. end.
The unit code:
Code: Pascal  [Select][+][-]
  1. unit mylib;
  2.  
  3. {$mode objfpc}{$H+}
  4.  
  5. interface
  6.  
  7. type
  8.         Tmylib_a = function (a: integer): integer; StdCall;
  9.         Tmylib_b = procedure (b: pinteger); StdCall;
  10.         Tmylib_c = function (c: pinteger): integer; StdCall;
  11.         Tmylib_d = function (d: pinteger): integer; StdCall;
  12.  
  13. var
  14.         mylib_a: Tmylib_a;
  15.         mylib_b: Tmylib_b;
  16.         mylib_c: Tmylib_c;
  17.         mylib_d: Tmylib_d;
  18.  
  19. procedure close_mylib();
  20. function init_mylib(path: string): integer;
  21.  
  22. implementation
  23.  
  24. uses dynlibs;
  25.  
  26. const
  27.         MYLIB_ERROR_ALREADY_OPEN =      1;
  28.         MYLIB_ERROR_CANNOT_OPEN =       2;
  29.         MYLIB_ERROR_MISSING_SYMBOL =    3;
  30.  
  31. var mylibhnd: TLibHandle;
  32.  
  33. procedure close_mylib();
  34. begin
  35.         if (mylibhnd <> DynLibs.NilHandle) then
  36.         begin
  37.                 FreeLibrary(mylibhnd);
  38.                 mylibhnd := DynLibs.NilHandle;
  39.         end;
  40. end;
  41.  
  42. function init_mylib(path: string): integer;
  43. begin
  44.         if (mylibhnd <> DynLibs.NilHandle) then
  45.         begin
  46.                 result := MYLIB_ERROR_ALREADY_OPEN;
  47.                 exit;
  48.         end;
  49.  
  50.         mylibhnd := LoadLibrary(path);
  51.         if (mylibhnd = DynLibs.NilHandle) then
  52.         begin
  53.                 result := MYLIB_ERROR_CANNOT_OPEN;
  54.                 exit;
  55.         end;
  56.  
  57.         mylib_a := Tmylib_a(GetProcedureAddress(mylibhnd, 'mylib_a'));
  58.         if (mylib_a = Tmylib_a(DynLibs.NilHandle)) then
  59.         begin
  60.                 close_mylib();
  61.                 result := MYLIB_ERROR_MISSING_SYMBOL;
  62.                 exit;
  63.         end;
  64.  
  65.         mylib_b := Tmylib_b(GetProcedureAddress(mylibhnd, 'mylib_b'));
  66.         if (mylib_b = Tmylib_b(DynLibs.NilHandle)) then
  67.         begin
  68.                 close_mylib();
  69.                 result := MYLIB_ERROR_MISSING_SYMBOL;
  70.                 exit;
  71.         end;
  72.  
  73.         mylib_c := Tmylib_c(GetProcedureAddress(mylibhnd, 'mylib_c'));
  74.         if (mylib_c = Tmylib_c(DynLibs.NilHandle)) then
  75.         begin
  76.                 close_mylib();
  77.                 result := MYLIB_ERROR_MISSING_SYMBOL;
  78.                 exit;
  79.         end;
  80.  
  81.         mylib_d := Tmylib_d(GetProcedureAddress(mylibhnd, 'mylib_d'));
  82.         if (mylib_d = Tmylib_d(DynLibs.NilHandle)) then
  83.         begin
  84.                 close_mylib();
  85.                 result := MYLIB_ERROR_MISSING_SYMBOL;
  86.                 exit;
  87.         end;
  88.  
  89.         result := 0;
  90. end;
  91.  
  92. end.
The main program's code:
Code: Pascal  [Select][+][-]
  1. program mytest;
  2.  
  3. {$mode objfpc}{$H+}
  4.  
  5. uses mylib, SysUtils;
  6.  
  7. function caller(ptr: pinteger): integer; cdecl;
  8. begin
  9.         ptr^ := ptr^ + 5;
  10.         result := ptr^ + 5;
  11. end;
  12.  
  13. const libname = './libmylib.so';
  14. var e, a, b, c, c2, d, d2: integer;
  15.  
  16. begin
  17.         e := init_mylib(libname);
  18.         if (e <> 0) then
  19.         begin
  20.                 WriteLn(StdErr, 'Cannot open "' + libname + ' - error ' + IntToStr(e) + '.');
  21.                 Halt(1);
  22.         end;
  23.  
  24.         a := mylib_a(9);
  25.         b := 10;
  26.         mylib_b(@b);
  27.         c := 11;
  28.         c2 := mylib_c(@c);
  29.         d := 12;
  30.         d2 := mylib_d(@d);
  31.         WriteLn(StdOut, 'a = ' + IntToStr(a) + ', b = ' + IntToStr(b) + ', c = ' + IntToStr(c) + ', c2 = ' + IntToStr(c2) + ', d = ' + IntToStr(d) + ', d2 = ' + IntToStr(d2));
  32.  
  33.         close_mylib();
  34. end.
Compiling the library:
Code: Bash  [Select][+][-]
  1. #!/bin/sh
  2. cd lib
  3. fpc -CX -Xs -XX mylib.pas
  4. rm mylib.o
  5. mv libmylib.so ../
Compiling the executable:
Code: Bash  [Select][+][-]
  1. #!/bin/sh
  2. fpc -CX -Xs -XX -k--export-dynamic mytest.pas
  3. rm mylib.o mylib.ppu mytest.o
  4. chmod +x mytest
This very same approach works if there is no calling of the main program's function, or if it is done by passing it's address to the library. Introducing the
Code: Pascal  [Select][+][-]
  1. type TCaller = function (ptr: pinteger): integer; cdecl;
  2.  
  3. var caller: TCaller; external name 'caller';
part resulted in LoadLibrary being unable to open the library. If i do `objdump -T libmylib.so`, the symbols are seem to be in their place and the library's name is correct too:
Code: [Select]
libmylib.so:     file format elf64-x86-64

DYNAMIC SYMBOL TABLE:
0000000000000ae0 g    DF .text  0000000000000005 mylib_d
0000000000000ab0 g    DF .text  0000000000000005 mylib_a
0000000000000ac0 g    DF .text  0000000000000005 mylib_b
0000000000000000      D  *UND*  0000000000000000 caller
0000000000000ad0 g    DF .text  0000000000000005 mylib_c

trev

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 1624
  • Former Delphi 1-7, 10.2 user
Re: Dynamically linked executables...possible?
« Reply #3 on: October 14, 2021, 01:44:43 am »
I don't have time to go through your code or understand the issue and you do not mention the OS, but you can look at what I wrote with code examples for macOS (UNIX) in this Wiki article.
Lazarus 2.3 + FPC 3.3.1 2021-10-19 macOS 10.14.6 Xcode 11.3.1
Lazarus 2.3 + FPC 3.3.1 2021-09-21 macOS 11.6 aarch64 Xcode 13
Lazarus 2.3 2021-08-11 FPC 3.2.2 FreeBSD 13.0 amd64 VMware VM
Lazarus 2.1 r61574 FPC 3.0.4 Ubuntu 20.04 Parallels VM
Lazarus 2.0.10 FPC 3.2.0 Win10 Parallels VM

y.ivanov

  • Sr. Member
  • ****
  • Posts: 283
Re: Dynamically linked executables...possible?
« Reply #4 on: October 14, 2021, 10:17:41 am »
*snip*
This very same approach works if there is no calling of the main program's function, or if it is done by passing it's address to the library. Introducing the
Code: Pascal  [Select][+][-]
  1. type TCaller = function (ptr: pinteger): integer; cdecl;
  2.  
  3. var caller: TCaller; external name 'caller';
part resulted in LoadLibrary being unable to open the library. If i do `objdump -T libmylib.so`, the symbols are seem to be in their place and the library's name is correct too:
Code: [Select]
libmylib.so:     file format elf64-x86-64

DYNAMIC SYMBOL TABLE:
0000000000000ae0 g    DF .text  0000000000000005 mylib_d
0000000000000ab0 g    DF .text  0000000000000005 mylib_a
0000000000000ac0 g    DF .text  0000000000000005 mylib_b
0000000000000000      D  *UND*  0000000000000000 caller
0000000000000ad0 g    DF .text  0000000000000005 mylib_c

Isn't it that the 'caller' variable was declared as pointer to fn in mylib, while the 'caller' in mytest is just a function, not a var?
Have you set the LD_LIBRARY_PATH env variable?

TCH

  • Full Member
  • ***
  • Posts: 147
Re: Dynamically linked executables...possible?
« Reply #5 on: October 14, 2021, 11:41:06 am »
I don't have time to go through your code or understand the issue and you do not mention the OS, but you can look at what I wrote with code examples for macOS (UNIX) in this Wiki article.
Thank you, but what you've covered in your article (creating a library and calling it's functions) is already working in my test setup. What does not is the "callback" mechanism, when the library calls a function of the dynamically linked executable. (The statical - where i simply give the function's address to the library - is working.)

Isn't it that the 'caller' variable was declared as pointer to fn in mylib, while the 'caller' in mytest is just a function, not a var?
It has to be a var which points to a function, since the function does not exists in the library. It works in C, if the executable is linked dynamically. This is in the C lib:
Code: C  [Select][+][-]
  1. extern int caller(int *ptr);
  2. int mylib_d(int *d);
  3.  
  4. int mylib_d(int *d)
  5. {
  6.         *d += 5;
  7.         return caller(d);
  8. }
And this is in the C executable:
Code: C  [Select][+][-]
  1. int caller(int *ptr)
  2. {
  3.         *ptr += 5;
  4.         return *ptr + 5;
  5. }
This approach also works in the statically linked Pascal executable, although i have to set that variable manually via a setter function of the library.
Have you set the LD_LIBRARY_PATH env variable?
No, but it is not needed in this case, as i used the full path to the library, not just it's name.

y.ivanov

  • Sr. Member
  • ****
  • Posts: 283
Re: Dynamically linked executables...possible?
« Reply #6 on: October 14, 2021, 01:08:24 pm »
*snip*

Isn't it that the 'caller' variable was declared as pointer to fn in mylib, while the 'caller' in mytest is just a function, not a var?
It has to be a var which points to a function, since the function does not exists in the library. It works in C, if the executable is linked dynamically. This is in the C lib:
Code: C  [Select][+][-]
  1. extern int caller(int *ptr);
  2. int mylib_d(int *d);
  3.  
  4. int mylib_d(int *d)
  5. {
  6.         *d += 5;
  7.         return caller(d);
  8. }
And this is in the C executable:
Code: C  [Select][+][-]
  1. int caller(int *ptr)
  2. {
  3.         *ptr += 5;
  4.         return *ptr + 5;
  5. }
This approach also works in the statically linked Pascal executable, although i have to set that variable manually via a setter function of the library.
Have you set the LD_LIBRARY_PATH env variable?
No, but it is not needed in this case, as i used the full path to the library, not just it's name.
I'm a bit confused!
Code: C  [Select][+][-]
  1. extern int caller(int *ptr);
Is not a variable definition. But:
Code: Pascal  [Select][+][-]
  1. var caller: TCaller; external name 'caller';
is.

Also, objdump -T mytest  doesn't show caller as a dynamic symbol.

Use:
Code: Pascal  [Select][+][-]
  1. exports caller;
into mytest.pas and:
Code: Pascal  [Select][+][-]
  1.     //type TCaller = function (ptr: pinteger): integer; cdecl;
  2.      
  3.     //var caller: TCaller; external name 'caller';
  4.    
  5.     function caller(ptr: pinteger): integer; cdecl; external name 'caller';
into mylib.pas.

Otherwise, you'll get an EAccessViolation.

TCH

  • Full Member
  • ***
  • Posts: 147
Re: Dynamically linked executables...possible?
« Reply #7 on: October 14, 2021, 01:59:32 pm »
I'm a bit confused!
Code: C  [Select][+][-]
  1. extern int caller(int *ptr);
Is not a variable definition. But:
Code: Pascal  [Select][+][-]
  1. var caller: TCaller; external name 'caller';
is.
You're right, i confused it. In the statical version it was.
Code: C  [Select][+][-]
  1. int (*caller)(int *ptr);
  2. int mylib_d(int *d);
  3. void mylib_set_caller(int (*caller_ptr)(int*));
  4.  
  5. int mylib_d(int *d)
  6. {
  7.         *d += 5;
  8.         return (*caller)(d);
  9. }
  10.  
  11. void mylib_set_caller(int (*caller_ptr)(int*))
  12. {
  13.         caller = caller_ptr;
  14. }
Also, objdump -T mytest  doesn't show caller as a dynamic symbol.
Weird. It did for me.
Use:
Code: Pascal  [Select][+][-]
  1. exports caller;
into mytest.pas and:
Code: Pascal  [Select][+][-]
  1.     //type TCaller = function (ptr: pinteger): integer; cdecl;
  2.      
  3.     //var caller: TCaller; external name 'caller';
  4.    
  5.     function caller(ptr: pinteger): integer; cdecl; external name 'caller';
into mylib.pas.

Otherwise, you'll get an EAccessViolation.
Thank you, it works perfectly!

y.ivanov

  • Sr. Member
  • ****
  • Posts: 283
Re: Dynamically linked executables...possible?
« Reply #8 on: October 14, 2021, 02:27:25 pm »
*snip*
Thank you, it works perfectly!
You're welcome!

Yet I'm wondering why such weird back-ref to the 'caller' used? I would pass the pointer to it as a parameter to the mylib_d().

TCH

  • Full Member
  • ***
  • Posts: 147
Re: Dynamically linked executables...possible?
« Reply #9 on: October 14, 2021, 05:57:40 pm »
*snip*
Thank you, it works perfectly!
You're welcome!

Yet I'm wondering why such weird back-ref to the 'caller' used? I would pass the pointer to it as a parameter to the mylib_d().
I am experimenting with libraries and i wanted to try out the dynamically linked executable setup too. I have a version where i pass the function's pointer to the library.

 

TinyPortal © 2005-2018