FPC 3.2.2 / Lazarus 2.2.6 patch for GLIBC_2.34 versioning issue

[WayneSherman]

So what was the final resolution of this?

I was one of the first people complaining about it https://forum.lazarus.freepascal.org/index.php/topic,61001.0.html and was reminded of the issue when wanting to do some rebuild-from-scratch stuff on Debian 12 today.

Looking at the standard trunk there's no mention of anything like -XLC, I can't find anything relevant in the bug tracker, and I don't see anything under https://www.freepascal.org/probs.html

I was thinking about this yesterday because I found a write-up explaining the inner workings of dso (dynamic shared object) symbol versioning:

Maintaining APIs and ABIs starting on page 34
https://www.akkadia.org/drepper/dsohowto.pdf

Both robert rozee and DonAlfredo did some good work on this issue as a proof of concept, but I don't think it got developed enough to turn into a patch or pull request to upstream.  The last work DonAlfredo did is in fpcupdeluxe commits in 2023-Oct:

https://github.com/LongDirtyAnimAlf/fpcupdeluxe/commits/master/?after=87cb49ff7446f13d57959399e7e925c7af00d494+0

He made a test build (Fpcupdeluxe 2.4.0bp) but it is no longer accessible.

[Fred vS]

Both robert rozee and DonAlfredo did some good work on this issue as a proof of concept

Robert Rozee mainly did the patch for fpc-ootb.
Note that the next version of libc, (if not using versioning) should not be compatible with fpc anymore because lot of things will be changed in the new table.

[Fred vS]

In case you are impatient, there is this (with also fpc 3.2.4 pre release).

I'm resigned rather than impatient, but I had to hunt out a Debian 11 system today to build a specific combination of FPC+Lazarus to run on Debian 12.

MarkMLl

Well, for FPC you could use fpc-ootb but for Lazarus, you will need to assign glibc2.2.5 versioning table to all the glibc methods used in Lazarus code.
There are a few, not a lot of.

[TRon]

So what was the final resolution of this?

As far as I am able to remember there initially was a bit of a resistance to address this issue with(in) the compiler but Robert mentioned that in due time there will be some advancement on that matter. Unfortunately I am able to find that statement as there are many topics mentioning the versioning issues.

At least there are possibilities to work around the issue (some good, some bad) most of them I was able to find in this 16 page long discussion.

I have not seen any mention of it in trunk either (but I could easily have overlooked).

[WayneSherman]

As far as I am able to remember there initially was a bit of a resistance to address this issue with(in) the compiler but Robert mentioned that in due time there will be some advancement on that matter. Unfortunately I am able to find that statement

When Alfred (DonAlfredo) was experimenting with fpc glibc symbol versioning, he was also building fpc using "-dFPC_USE_LIBC".  This caused an fpc bug (unrelated to glibc symbol versioning) to show up while trying to build the Lazarus code base (specifically fpdebug was using a kernel syscall, but the system call functions were not being exported when FPC_USE_LIBC was defined).  Apparently the default (for x86_64 Linux) is for the fpc RTL to use system calls and not glibc so this bug wasn't found until 2023-Oct (EDITED).

I don't understand the reason for using "-dFPC_USE_LIBC" to build fpc.  To build an fpc compiler with the widest compatibility should it use kernel system calls and no glibc, or should it use no system calls and only glibc?  (EDITED - I wonder if using kernal syscalls only and no glibc is preferred for better forward/backward compatibility.  In any case if glibc calls are made, these should be declared with base symbol versions)

More info here:
https://github.com/LongDirtyAnimAlf/fpcupdeluxe/issues/634
(the statement you might be looking for is at the bottom)

and here:
https://gitlab.com/freepascal.org/lazarus/lazarus/-/issues/40575

[robert rozee]

as an aside, the very first posting to this thread (https://forum.lazarus.freepascal.org/index.php/topic,64731.msg492467.html#msg492467) contains a complete explanation on how to apply the patch. the only source code change is to the function pd_external() that is contained in the file /usr/share/fpcsrc/3.2.2/compiler/pdecsub.pas.

there is more than enough information provided to allow the same change to be made to later versions of lazarus/FPC. someone just needs to create a small 'patcher program' that is capable of scanning through /usr/share/fpcsrc/3.2.2/compiler/pdecsub.pas to find the two places in pd_external() where the relevant blocks of code need to be added. this functionality could indeed be added as a further function to the MakeBase utility i included.

it has been a year since i last looked at all this, and my memory of many of the finer details has faded - do not worry, i've always had a bad memory. it is now up to someone else to proceed forward, if indeed there is any will to do so.


cheers,
rob   :-)

[MarkMLl]

Both robert rozee and DonAlfredo did some good work on this issue as a proof of concept, but I don't think it got developed enough to turn into a patch or pull request to upstream.  The last work DonAlfredo did is in fpcupdeluxe commits in 2023-Oct:

They did an enormous amount of good work, and I'm disturbed that the issue still seems to be ignored by the overall FPC project.

MarkMLl

[MarkMLl]

there is more than enough information provided to allow the same change to be made to later versions of lazarus/FPC. someone just needs to create a small 'patcher program' that is capable of scanning through /usr/share/fpcsrc/3.2.2/compiler/pdecsub.pas to find the two places in pd_external() where the relevant blocks of code need to be added. this functionality could indeed be added as a further function to the MakeBase utility i included.

Yes, I think I'm going to have to look at that.

I've got a well-developed sequence of building FPC and then Lazarus from scratch on systems with only a very basic OS which has served me since the earliest public versions: I've still got a system here with 2.2.2 which I use for testing GTK1 stuff. I can cope with not being able to e.g. install older than 2.6.4 on x64 because the target doesn't exist, but feel that it's highly undesirable for there to be other "inflection points" where a native build is problematic: yesterday I had to dig out a Debian 11 system to regenerate the combination of Lazarus 2.0.6 with FPC 3.0.4 because I couldn't get there either forwards or backwards on Debian 12.

Having said which, I'm an extremely satisfied user of FpcUpDeluxe for the Pico etc.

Just to keep them in the Symbol Versioning record:

More info here:
https://github.com/LongDirtyAnimAlf/fpcupdeluxe/issues/634
(the statement you might be looking for is at the bottom)

and here:
https://gitlab.com/freepascal.org/lazarus/lazarus/-/issues/40575

MarkMLl

[marcov]

I don't understand the reason for using "-dFPC_USE_LIBC" to build fpc.  To build an fpc compiler with the widest compatibility should it use kernel system calls and no glibc, or should it use no system calls and only glibc?

In theory and according to most linux/glibc sources it should be glibc. But that is a bit of a knee jerk reaction that quite often doesn't take into account that Pascal sources won't automatically pick up changed headers and also don't take into account routinely having some cross distro/version binary compatibility.  I.e. they don't consider FPC specific circumstances.

On the other hand, one could argue that the current syscall situation is an artifact of the years 2000-2005 when glibc was going through changes (most notably 64-bit file support, also on the then mostly 32-bit systems)

To see what is better probably requires both systems to be used in parallel for a while, and honestly looking what causes less problems/breakage.

[WayneSherman]

In theory and according to most linux/glibc sources it should be glibc. But that is a bit of a knee jerk reaction that quite often doesn't take into account that Pascal sources won't automatically pick up changed headers and also don't take into account routinely having some cross distro/version binary compatibility.  I.e. they don't consider FPC specific circumstances.

On the other hand, one could argue that the current syscall situation is an artifact of the years 2000-2005 when glibc was going through changes (most notably 64-bit file support, also on the then mostly 32-bit systems)

To see what is better probably requires both systems to be used in parallel for a while, and honestly looking what causes less problems/breakage.

The Linux kernel policy is to never break user space (i.e. new kernel versions maintain backward compatibility with existing compiled software):
https://unix.stackexchange.com/questions/235335/why-is-there-a-linux-kernel-policy-to-never-break-user-space

glibc, using symbol versioning, also strives for new glibc releases to maintain backward compatibility with previously compiled software that uses the library:
https://developers.redhat.com/blog/2019/08/01/how-the-gnu-c-library-handles-backward-compatibility

I agree that testing would be needed and experience would have to be gained using both approaches before deciding which is generally more backward compatible.

In the case of fpc and the rtl using the glibc library on Linux (x86_64):
The pascal header conversion and code that calls library functions and code that references external symbols was written and tested against a specific version of glibc with specific documented behavior for those functions and symbols.

1. Do you think the fpc and rtl source code declarations should be specifically pinned to those versions of the glibc functions they were originally written to work with? (i.e. declare the specific glibc SONAME which includes the major version and declare specific versions of the external symbols in the fpc/rtl source code)

2. Do you think the fpc and rtl glibc source code declarations should not specify any versions and allow the compiled binaries to get specific .so version and symbol versions from the static linker at build time (which uses the glibc library present on the system at build time)?

or
3. Should there be the ability to choose between the two options above when compiling fpc/rtl?

(EDITED to clarify that the library SONAME includes the major version in #1 above)

[Fred vS]

In theory and according to most linux/glibc sources it should be glibc. But that is a bit of a knee jerk reaction that quite often doesn't take into account that Pascal sources won't automatically pick up changed headers and also don't take into account routinely having some cross distro/version binary compatibility.  I.e. they don't consider FPC specific circumstances.

On the other hand, one could argue that the current syscall situation is an artifact of the years 2000-2005 when glibc was going through changes (most notably 64-bit file support, also on the then mostly 32-bit systems)

To see what is better probably requires both systems to be used in parallel for a while, and honestly looking what causes less problems/breakage.

The Linux kernel policy is to never break user space (i.e. new kernel versions maintain backward compatibility with existing compiled software):
https://unix.stackexchange.com/questions/235335/why-is-there-a-linux-kernel-policy-to-never-break-user-space

glibc, using symbol versioning, also strives for new glibc releases to maintain backward compatibility with previously compiled software that uses the library:
https://developers.redhat.com/blog/2019/08/01/how-the-gnu-c-library-handles-backward-compatibility

I agree that testing would be needed and experience would have to be gained using both approaches before deciding which is generally more backward compatible.

In the case of fpc and the rtl using the glibc library on Linux (x86_64):
The pascal header conversion and code that calls library functions and code that references external symbols was written and tested against a specific version of glibc with specific documented behavior for those functions and symbols.

1. Do you think the fpc and rtl source code declarations should be specifically pinned to those versions of the glibc functions they were originally written to work with? (i.e. declare the specific glibc .so version and the specific versions of the external symbols in the fpc/rtl source code)

2. Do you think the fpc and rtl glibc source code declarations should not specify any versions and allow the compiled binaries to get specific .so file version and symbol versions from the static linker at build time (which uses the glibc library present on the system at build time)?

or
3. Should there be the ability to choose between the two options above when compiling fpc/rtl?

In case you asked it to me too  , I will vote for 1. for Linux amd64 and assign GLIBC_2.2.5 for each libc method used by fpc.
Beause fpc was created using this table and never updated for new table and mainly because I did test it deeply and it works perfectly.

[Ștefan-Iulian Alecu]

In case you asked it to me too  , I will vote for 1. for Linux amd64 and assign GLIBC_2.2.5 for each libc method used by fpc.
Beause fpc was created using this table and never updated for new table and mainly because I did test it deeply and it works perfectly.

How's holding off to a 18 year old version of glibc good?

In the case of fpc and the rtl using the glibc library on Linux (x86_64):
The pascal header conversion and code that calls library functions and code that references external symbols was written and tested against a specific version of glibc with specific documented behavior for those functions and symbols.

1. Do you think the fpc and rtl source code declarations should be specifically pinned to those versions of the glibc functions they were originally written to work with? (i.e. declare the specific glibc .so version and the specific versions of the external symbols in the fpc/rtl source code)

2. Do you think the fpc and rtl glibc source code declarations should not specify any versions and allow the compiled binaries to get specific .so file version and symbol versions from the static linker at build time (which uses the glibc library present on the system at build time)?

or
3. Should there be the ability to choose between the two options above when compiling fpc/rtl?

What do other compilers do in this regard? I feel like no. 3 would be the best option, but by default have the behavior of no. 2.

[Fred vS]

In case you asked it to me too  , I will vote for 1. for Linux amd64 and assign GLIBC_2.2.5 for each libc method used by fpc.
Beause fpc was created using this table and never updated for new table and mainly because I did test it deeply and it works perfectly.

How's holding off to a 18 year old version of glibc good?

Because the libc developers promised that each new version would integrate the old tables and if they are defined for the link the new versions will assume them. And as serious developers, they keep their word.

[Fred vS]

2. Do you think the fpc and rtl glibc source code declarations should not specify any versions and allow the compiled binaries to get specific .so file version and symbol versions from the static linker at build time (which uses the glibc library present on the system at build time)?

Sorry to be the boring ones again but there is no specific .so file for libc.
There is libc.so.6 and each new version has still the same name libc.so.6 but with new symbols tables.
It is the reason why signed symbol tables should always be used for each method.
And the magic of libc.so.6 is that if you choose a old table compatible with your program, libc.so.6 will adapt the last updated method to be compatible with that old table.
In the case you want to use different tables, it will ask lot of work for fpc to always update his code, each time a new table is added in libc.so.6.

Imho, better trust in the loyalty of libc developers.

[WayneSherman]

In case you asked it to me too  , I will vote for 1. for Linux amd64 and assign GLIBC_2.2.5 for each libc method used by fpc.
Beause fpc was created using this table and never updated for new table and mainly because I did test it deeply and it works perfectly.

How's holding off to a 18 year old version of glibc good?

Linking to the older symbols doesn't mean you get old, buggy, or insecure implementations.  The symbol versions are intended to provide a stable interface (API/ABI) for backwards compatibility.  The underlying implementations still get bug and security fixes.