Dynamic Library on Linux

[Kevan S. Hashemi]

The loadlibrary method does look more reliable, especially with the addition of ExtractFilePath(ParamStr(0)), which looks useful. Fred: thanks for your code. I will try this out on Windows, Linux, and MacOSX. Best, Kevan

[Fred vS]

I will try this out on Windows, Linux, and MacOSX.

Mac OSX uses bundle for application.

https://developer.apple.com/library/archive/documentation/CoreFoundation/Conceptual/CFBundles/BundleTypes/BundleTypes.html#//apple_ref/doc/uid/10000123i-CH101-SW1

You just have to add your libraries in the Ressource-files directory of the bundle.
Those Apple Ressource files can only be loaded by dynamic loading, so the good news is that LoadLibrary does it.

So, with MacOSX, it could be something like:

MyLib := LoadLibrary(ExtractFilePath(ParamStr(0)) + '/Resources/libd.dylib');

[EDIT]  See next post of trev for good explanations.

Fre;D

[Kevan S. Hashemi]

Fred,

I like your solution for MacOSX.

The linking problem in practice has another complication. Suppose we have a generic program, call it 'prog', that we don't write ourselves, but which we want to augment with our own libraries. We're expecting 'prog' to be able to load our dynamic library.

But prog knows nothing of our library when prog is compiled. Prog cannot call any routine in our library unless our library provides a routine with a pre-defined format, with a name that can be deduced from the library file name. So prog might say that libd.so must provide a routine D_Init that takes a pointer and returns an integer.

Now prog loads libd.so and calls D_Init. The job of D_Init is to install in prog a list of new commands that prog can execute. To do that, D_Init must call an "install command" routine defined by prog itself, call it Prog_InstallCmd, which takes a pointer and a name for the command. So now we see that libd.so must be linked against a library prog.so that defines the structure of Prog_InstallCmd.

On Linux, I think we do this (have not tested fully yet):

fps -k-lprog d

On MacOSX, I'm doing the following to link libd.dylib with the TclTk frameworks, where I'm running fpc in a directory adjacent to Prog.app, the application bundle (fully tested):

fpc -k"-F../Prog.app/Contents/Frameworks -framework Tcl -framework Tk" d

Best, Kevan

[Fred vS]

Hello Kevan.

Sorry, but I did not catch the problem.

When you use external your "prog" must also know the names of the methods of the linked library.

Sure you want to demonstrate something else but I dont catch what.

[Fred vS]

Do you want to know the method names of a unknown library?

If it is that, you can retrieve it on Unix system with nm:

Code: Pascal  [Select][+][-]

1. $ nm  libunknown.so

So you may imagine in your code, via a TProcess, retrieve the name of the method but I dont know how to retrieve the parameters used by the exported method.

Aslo I dont know to determine if a exported method is void (procedure) or not (function).

[Fred vS]

Hum, I think I get it (maybe).

You want to have a exported method that has the same name "init_lib" and same parameter, for each library, and when the program load a new library, he first call "init_lib".

That "init_lib" method will add all the stuffs needed by the prog to deal with that new library.

Ha, ok, nice and good to experiment.

[Kevan S. Hashemi]

I'm not sure I understand my own point now, it seemed so profound at the time. I end up with two libraries that call routines both ways. One library I make, the other is pre-existing and knows nothing about the library I'm making. The only way to get that to work is to use the dynamic loading scheme you recommend: where you load at run-time, under instruction from the user, and specify the name of the library you want to load. But we don't need to use the command-line "-k" flag to sort this out, we can sort it out with the various directives and identifiers available in the Pascal code.

[Fred vS]

Hello Kevan.

Hum, maybe I did not understand too.

I was thinking that "prog" was already compiled, with instruction to load some libraries (plugins?) from a directory.
After loading, "prog" call a method "lib_init" from each library that will add all what needed for the library of the "plugins" directory.

But now it seems that you want to re-compile "prog" with "-k"  flag.

I am lost, sorry.

[Fred vS]

we can sort it out with the various directives and identifiers available in the Pascal code.

Sure.

I dont know what kinds of method you want to export, must it be compatible with C applications?

If you use "universal" variables, like integer or pchar, with few type of methods, it can be predefined with type in the "prog" code.

[trev]

So, with MacOSX, it could be something like:

Code: Pascal  [Select][+][-]

1. MyLib := LoadLibrary(ExtractFilePath(ParamStr(0)) + '/Resources/libd.dylib');

No, no, no 

1. For a start, any library in an application bundle should be in Project1.app/Contents/Resources/Frameworks
2. Never use ExtractFilePath(ParamStr(0)) in any UNIX or UNIX-like OS, it only leads to tears.
3. The correct method to use so that the executable can find the library is:

Code: Pascal  [Select][+][-]

1. install_name_tool -add_rpath "@executable_path/../Frameworks/." project1

I recently wrote fairly extensive Wiki articles: macOS Dynamic Libraries along with macOS Static Libraries which explains it all (I hope).

[Fred vS]

So, with MacOSX, it could be something like:

Code: Pascal  [Select][+][-]

1. MyLib := LoadLibrary(ExtractFilePath(ParamStr(0)) + '/Resources/libd.dylib');

No, no, no 

1. For a start, any library in an application bundle should be in Project1.app/Contents/Resources/Frameworks
2. Never use ExtractFilePath(ParamStr(0)) in any UNIX or UNIX-like OS, it only leads to tears.
3. The correct method to use so that the executable can find the library is:

Code: Pascal  [Select][+][-]

1. install_name_tool -add_rpath "@executable_path/../Frameworks/." project1

I recently wrote fairly extensive Wiki articles: macOS Dynamic Libraries along with macOS Static Libraries which explains it all (I hope).

Ok, for my post about Mac OSX, I dont have a Mac, I should not add that post.

But for that:

2. Never use ExtractFilePath(ParamStr(0)) in any UNIX or UNIX-like OS, it only leads to tears.

Sorry but I  never cried using this on Linux or FreeBSD.

[EDIT]
From https://www.freepascal.org/docs-html/rtl/system/paramstr.html

In the interest of portability, the ParamStr function tries to behave the same on all operating systems: like the original ParamStr function in Turbo Pascal. This means even on Unix, paramstr(0) returns the full path to the program executable. A notable exception is Mac OS X, where the return value depends on how the application was started. It may be that just the name of the application is returned (in case of a command-line launch)

So, ok for Mac OS X, but for the others (the topic is about Dynamic Library on Linux) ?

[Kevan S. Hashemi]

My point is merely something I found interesting, and probably nothing new to any of you. It applies to any operating system. I don't expect you to keep reading, but I feel it my duty to attempt to be clear rather than vague.

With FPC's LoadLibrary command, we can write a master program that allows the user to select a dynamic library to load. So we load the library, call it the "analysis library". Then what? We trust that the analysis library contains a routine called Init, and the master program runs that routine. The Init routine calls a routine made available by the master program's own dynamic library, a routine like "install_new_command", and installs a bunch of new analysis commands. which the user can now run.

Thus the analysis library declares "install_new_command" as an external, and it makes available "Init" as an export. So far as I can tell, using the "LoadLibrary" command in the master program is the only way to make this happen. So I'm grateful to Fred for showing me how to use LoadLibrary.

It turns out that, on Linux, we can compile the analysis library without linking to the library that declares "install_new_program" being present. That makes sense to me. The compiler trusts that, when the analysis library is loaded, the "install_new_program" routine will be available for dynamic linking. If it isn't, the the program will crash.

But on MacOSX, when we compile the analysis library, fpc wants to see the library declaring "install_new_command", or else it aborts with an error. That I don't understand. But I'm looking into it, and that's a subject for the MacOSX board.

[Fred vS]

The compiler trusts that, when the analysis library is loaded, the "install_new_program" routine will be available for dynamic linking. If it isn't, the the program will crash.

Exactly!

But you may have a filter like this to not crash the program:

       

Code: Pascal  [Select][+][-]

1. MyMethod := TMyMethod(GetProcedureAddress(MyLib, 'install_new_program'));
2.        if MyMethod = nil then writeln('No install_new_program routine found');

But on MacOSX,

For this, I better shut up otherwise some people will cry...

[trev]

But for that:

2. Never use ExtractFilePath(ParamStr(0)) in any UNIX or UNIX-like OS, it only leads to tears.

Sorry but I  never cried using this on Linux or FreeBSD.

[EDIT]
From https://www.freepascal.org/docs-html/rtl/system/paramstr.html

In the interest of portability, the ParamStr function tries to behave the same on all operating systems: like the original ParamStr function in Turbo Pascal. This means even on Unix, paramstr(0) returns the full path to the program executable. A notable exception is Mac OS X, where the return value depends on how the application was started. It may be that just the name of the application is returned (in case of a command-line launch)

So, ok for Mac OS X, but for the others (the topic is about Dynamic Library on Linux) ?

To quote Jonas:

Paramstr(0) must only be used on Dos and Windows (and maybe OS/2). On all other platforms the results are unpredictable and it will not work as you expect (in the best case it will never work; in the worst case it will only fail under certain circumstances, resulting in hard-to-find bugs). Read the mailing list thread I quoted above for more information.

Jonas details the "why" in this mailing list thread post.

[Fred vS]

@trev.

OK, thanks for the info (from 2009), and so, what do you propose as alternative to ParamStr(0)?