FPC 3.2.2 / Lazarus 2.2.6 patch for GLIBC_2.34 versioning issue

[dbannon]

Of course, the need to be root only applies if fpc/lazarus are installed in root space. Lots and lots of user with Lazarus in user space and lots with fpc there too.

But that aside, congratulations, brilliant piece of work !

Davo

[Tomas Hajny]

Out of curiosity, I looked very quickly at the patches discussed in this thread and I noticed at least two things which should be certainly done differently (note that this doesn't touch the primary intentions - I don't feel to be in the position to judge that part):

1) You should better not "reuse" fields in the PPU unit flags for a different meaning even if the previous use was discontinued.

2) The patch is restricted to x86_64-linux. I don't know the reasons, but it doesn't feel right at all. As long as the problem being solved exists also on other CPUs and probably also other operating systems, restricting the patch to just one particular combination, although possibly the one used most widely, doesn't look like a good idea.

[robert rozee]

2) The patch is restricted to x86_64-linux. I don't know the reasons, but it doesn't feel right at all. As long as the problem being solved exists also on other CPUs and probably also other operating systems, restricting the patch to just one particular combination, although possibly the one used most widely, doesn't look like a good idea.

my version of the patch is just linux specific, although not tested against 32-bit or non x86. the symbol BASE version numbers are architecture-specific and so retrieved from the local libc and stored in a file placed into /tmp. on the whole, adapting to any other unix-like OS should be trivial, but again not tested; the problem i was trying to fix arose out of my use of Lazarus/FPC on 64-bit x86 Linux.

btw, your comments are appreciated. please do give the patch a try!


cheers,
rob   :-)

[DonAlfredo]

I have made a parser of glibc versions for all major cpu.
See screenshot.
Will try to use it for then versioning patch.

[WayneSherman]

I have made a parser of glibc versions for all major cpu.
See screenshot.
Will try to use it for then versioning patch.

Very nice.  Are you parsing the version information from the *.abilist files in the libc source git repository?
(I was thinking of making a converter using pascal or bash script to create the glibc_x64.inc file from the abilist files)

[DonAlfredo]

Yes !!
Found the link in this thread. Did not know before.
More cpu can be added, as well as other versioned symbols.
Now thinking of a way to easy implement into the compiler.

[WayneSherman]

Now thinking of a way to easy implement into the compiler.

I think these things should be considered:
1) Ability to turn on and off external declaration versioning globally during compilation (-XLC accomplishes that)
2) Allow the versioning to be bypassed in source code. (if one wants to NOT version a particular external declaration how do we indicate it will be skipped)
3) Ability to override the version mapping.  Compiling the mapping table into the compiler is convenient (i.e. no external map files needed), but not very flexible.  If one wants to override the built-in versioning of some or all symbols it might be good to allow a "map" file to be passed as a command line parameter that can override one or more of the built in version entries.

[DonAlfredo]

Good tips !
Bypassing should already be honored: just add @GLIBC.... into the external function call declaration.
At this moment, an override is not necessary. This patch/option will have a very small user base. For these users, it will do exactly what is needed/wanted.
One of the major things to test is Lazarus itself. Lazarus will (also) be built with the patched RTL.
I will try to implement this into fpcupdeluxe but only for FPC stable (3.2.2). And lets wait and see how it works out there.

[WayneSherman]

Bypassing should already be honored: just add @GLIBC.... into the external function call declaration.

Let's say I am working with a library provided by others that has a "wait" procedure, how do I prevent symbol versioning being added?:

Code: Pascal  [Select][+][-]

1. procedure wait(const msTime: Double); cdecl; external a-library.so name 'wait';

At this moment, an override is not necessary.

Agreed.  Save that for a future time if it is deemed necessary.

This patch/option will have a very small user base.

Why do think that? Who does not want their compiled binaries to run on the largest number of systems both old to new? Who prefers to maintain an old Linux distro for the purpose of building and linking binaries when all the development and testing was done a new distro?  Once it is established that this solution works well, I think it should be merged into upstream FPC sources and everyone that wants can use it (which I predict will be most developers).

I will try to implement this into fpcupdeluxe but only for FPC stable (3.2.2). And lets wait and see how it works out there.

Agreed.  It is better to test on a non-moving target.

[DonAlfredo]

Update.
After making everything work, I was able to successfully compile a legacy version of a simple form on the latest Arch Linux. All calls towards glibc were found and converted into versioned calls (mostly 2.2.5). So far so good.

However.
Running this app on Ubuntu 18 failed miserably. The call to dlopen failed.
Turns out that this has happened.
https://sourceware.org/glibc/wiki/Release/2.34#Libraries_merged_into_libc
So, no libdl.so.2 any more on these modern systems. My app was indeed only linked against libc.so.6 on the latest Arch Linux. The dlopen function is located in libc.so.6
And on Ubuntu 18 a function like dlopen is still located in libdl.so.2, resulting in the error "dlopen not found".

Sidenote.
I did not succeed in forcing the linker to link against an more or less empty libdl.so.2
(use --no-as-needed linker option)

Any ideas are very welcome !!

Update.
Forgot that the linker looks for static libs by default.
This works to get the unneeded libs into the executable:
--no-as-needed -l:libdl.so.2 -l:librt.so.1

[robert rozee]

[...] However.
Running this app on Ubuntu 18 failed miserably. The call to dlopen failed.
Turns out that this has happened.
https://sourceware.org/glibc/wiki/Release/2.34#Libraries_merged_into_libc
So, no libdl.so.2 any more on these modern systems. My app was indeed only linked against libc.so.6 on the latest Arch Linux. The dlopen function is located in libc.so.6
And on Ubuntu 18 a function like dlopen is still located in libdl.so.2, resulting in the error "dlopen not found".

that one caught me out too when testing the patch on a new VM. i'd forgotten an important step:

create a symlink from libdl.so to libdl.so.2 using:

Code: Bash  [Select][+][-]

1. sudo ln -s /lib/x86_64-linux-gnu/libdl.so.2 /lib/x86_64-linux-gnu/libdl.so

the above symlink is really important to ensure libdl.so.2 is correctly referenced in generated ELF headers.


cheers,
rob   :-)

[WayneSherman]

For future reference, the libc release notes regarding the merging of libraries:
https://sourceware.org/pipermail/libc-alpha/2021-August/129718.html

* In order to support smoother in-place-upgrades and to simplify
  the implementation of the runtime all functionality formerly
  implemented in the libraries libpthread, libdl, libutil, libanl has
  been integrated into libc.  New applications do not need to link with
  -lpthread, -ldl, -lutil, -lanl anymore.  For backwards compatibility,
  empty static archives libpthread.a, libdl.a, libutil.a, libanl.a are
  provided, so that the linker options keep working.  Applications which
  have been linked against glibc 2.33 or earlier continue to load the
  corresponding shared objects (which are now empty).  The integration
  of those libraries into libc means that additional symbols become
  available by default.  This can cause applications that contain weak
  references to take unexpected code paths that would only have been
  used in previous glibc versions when e.g. preloading libpthread.so.0,
  potentially exposing application bugs.

[marcov]

the above symlink is really important to ensure libdl.so.2 is correctly referenced in generated ELF headers.

That symlink should be installed by glibc-devel or whatever the base development package on your distro is.

[robert rozee]

the above symlink is really important to ensure libdl.so.2 is correctly referenced in generated ELF headers.

That symlink should be installed by [...]

should be, but is not. i recall this being argued about some time back. now if FPC accepted numeric extensions to after the .so then this wouldn't be such a problem!


cheers,
rob   :-)

[DonAlfredo]

Well, I guess that the compat-app should not rely on dev-packages to be installed on the legacy system itself. That is why it should link against the versioned libs. And I hope these versions are more or less stable for the last [15] years.