Building FPC 3.2.2 on Debian "Testing" (Bookworm) x86_64

[marcov]

This is ENTIRELY about 3.0.4 built from source, and its inability to build 3.2.2 on one particular platform.

That should have worked.

No it SHOULDN'T, only the previous version (i.e 3.2.0) is a guarantee, and even that is sometimes doubtful for some new targets (requiring cross compilation to bootstrap).

The second required version (3.0.4 in this case)  in the makefile is only there to ease the release process (when the supported compiler changes) and for certain targets that skipped the previous release or where it was defunct somehow. It is BY NO MEANS A GUARANTEE.

Or if it didn't work, it should have provided some sensible error message.

We can't retroactively change a released and obsolete 3.0.4 to create proper error messages.

[MarkMLl]

Nobody is saying you need to fix 3.0.4. You need to find what in 3.2.x is being broken by the latest Debian.

Look, I've been a committed FPC user for what- 15 years or so?- but this is testing my patience to the limit. You need to step back and consider how it would look to a potential new user: whether a refugee from Delphi, or somebody who's never touched Pascal before.

MarkMLl

[marcov]

Nobody is saying you need to fix 3.0.4. You need to find what in 3.2.x is being broken by the latest Debian.

It isn't apparently as you say yourself:

I can confirm that on Debian "Bookworm" x86_64 FPC 3.2.0 will build 3.2.2 successfully.

Look, I've been a committed FPC user for what- 15 years or so?- but this is testing my patience to the limit. You need to step back and consider how it would look to a potential new user: whether a refugee from Delphi, or somebody who's never touched Pascal before.

You are working with an experimental version of the OS that will not be released until 2nd half 2023. That is not beginner refugee territory.

So let's reboot the discussion. Is your quote above that 3.2.0 bootstrap works correct ?

[TRon]

You are working with an experimental version of the OS that will not be released until 2nd half 2023. That is not beginner refugee territory.

A bit of a sidetrack (apologies) but I (for one) do not have a choice. There isn't a single (older) distribution that runs on this specific hardware unless I'm willing to custom compile the kernel (running a testing release is already new to me).

Although I am able to understand the discussion wrt the correct starting compiler I do believe there are bigger fish to fry ?

Am I correct when I state that the only variable in creating the compiler are the binutils (or am I missing something else there) ? (*)

(*) which fwiw is version 2.40

[marcov]

You are working with an experimental version of the OS that will not be released until 2nd half 2023. That is not beginner refugee territory.

A bit of a sidetrack (apologies) but I (for one) do not have a choice. There isn't a single (older) distribution that runs on this specific hardware unless I'm willing to custom compile the kernel (running a testing release is already new to me).

Although I am able to understand the discussion wrt the correct starting compiler I do believe there are bigger fish to fry ?

Like? Which post summarizes your exact problem description when bootstrapping with 3.2.0 ?

Am I correct when I state that the only variable in creating the compiler are the binutils (or am I missing something else there) ? (*)

(*) which fwiw is version 2.40

Including possible modifications for the distribution (iow, is it vanilla 2.40?) I'm not Linux expert (anymore), but iirc there are also additional binaries like crti.o and crtn.o that come iirc with glibc-devel.   Easiest is to try compile something and use -s to view link.res and ppas.sh. Once for a binary that links  to libc, and once not.

And of course the binary loader might also be different or have new requirements. (e.g. requiring some new section, a certain identifier etc)

[MarkMLl]

Like? Which post summarizes your exact problem description when bootstrapping with 3.2.0 ?

I'm stepping back from the forum for a while, I've had more than enough of this.

PMs are set to send me an email alert.

MarkMLl

[dbannon]

You are working with an experimental version of the OS (Testing) that will not be released until 2nd half 2023. That is not beginner refugee territory.

Thats possibly not quite accurate Marcov, many people use Testing, especially this late in its release cycle. Ubuntu and many other Linux Distribution are built as snapshots of Testing. I suspect you are thinking of 'unstable' or 'sid' - they sure are experimental. Testing has quite high standards and any package with problems in Testing is removed.

Importantly, Testing, Bookworm at present, is what will be 'stable' before too long. If there is a problem there, we need to identify it and either work around it or report a bug. Once its released, it will be too late.

Davo

[marcov]

Thats possibly not quite accurate Marcov, many people use Testing, especially this late in its release cycle.

Ubuntu and many other Linux Distribution are built as snapshots of Testing. I suspect you are thinking of 'unstable' or 'sid' - they sure are experimental. Testing has quite high standards and any package with problems in Testing is removed.

Importantly, Testing, Bookworm at present, is what will be 'stable' before too long. If there is a problem there, we need to identify it and either work around it or report a bug. Once its released, it will be too late.

In summary:

•   Mark has problems with building 3.2.2 with 3.0.4, a combination based on what seems to be allowed in the makefile. I say that is not an official route.  (though it is one that in general is quite commonly  doable, it was never guaranteed). See e.g. decade old buildfaq. In all cases: if you have problems building try the guaranteed working solution (full previous release install, and then bootstrap using the release source)
• TRon reports problems, but confirms 3.2.2 builds with 3.2.0, which is the supported way. I'm still waiting for his summary for what is the actual problem

Sometimes it can help to pass -dFPC_USE_LIBC to temporarily generate a (g)libc using compiler binary using e.g. make cycle, and only then try to bootstrap with that glibc ppc386/x64 without -dFPC_USE_LIBC.

As said before, running with -s and then manually run ppas.sh can sometimes be insightful too, and allow multiple link attempts with slightly different options (in case it has something to do with the -T stuff, which is something done for either 3.2.0 or 3.2.2).

Speaking of which; TRon:  the linker (LD) also has some default linker scripts internally that might have changed. The -T change was something about that, with in the distance past the script on the commandline being a replacement, and later only an addition (or the other way around).  But it is more likely to appear with Lazarus binaries that use resources, as most of the script changes are related to that.

[TRon]

Like? Which post summarizes your exact problem description when bootstrapping with 3.2.0 ?

Sorry marcov. That was a remark from my hand that you obviously took into context of this thread. I meant on a more general note e.g.current bugs, new release etc. I do not have a problem bootstrapping with 3.2.0.

I understand that it is not a concern of the team that older compilers won't build (however frustrating that might be for some).

Thank you for your elaborated posts on my question regarding external factors that influences (or can influence) the build-process. I gladly take them into consideration in trying to figure out what goes wrong for the older builds.

Whatever the culprit, it seems to have been addressed and/or fixed with 3.20 although I am aware it could be pure coincidence (which, if the case, is what concerns me more  ).

@MarkMLI:
Sorry to see that this discussion is upsetting you enough to turn away (if even for a while). That was not my intention when I bumped this thread. I just wanted to share my experience so that it won't come to a surprise (to others as well) in a coupe of months.  No problem if you do not (wish to) react (for the time being).

[dbannon]

Whatever the culprit, it seems to have been addressed and/or fixed with 3.20 although I am aware it could be pure coincidence (which, if the case, is what concerns me more  ).

TRon, there are a lot of changes between compiler versions, otherwise there would not need to be new releases. So, rather than looking for a "culprit" you should be chalking up a win if you can do an unexpected build with a compiler earlier than the recommended one. But its a win by accident, not intentional, the devs don't try and support earlier compilers, if it happens, its a case of "gee, thats handy".

Davo

[marcov]

Like? Which post summarizes your exact problem description when bootstrapping with 3.2.0 ?

Sorry marcov. That was a remark from my hand that you obviously took into context of this thread. I meant on a more general note e.g.current bugs, new release etc. I do not have a problem bootstrapping with 3.2.0.

Good. But then it is out of the hands of the FPC team nowadays. If the bootstrap stuff needs occasional maintenance to old branches that is done by them themselves.

I understand that it is not a concern of the team that older compilers won't build (however frustrating that might be for some).

The problem is that Linux binary standards are a (very slowly) moving target. An old source is however static, and there are complications like that source being tied to old tools (LD,AR,AS and make chiefly, but there are more detail dependencies).  Later versions reduce the dependency on AS/AR more and more. (using pascal startup code and an inernal assembler).

The logical way would to periodically cross bootstrap from a binary stable target like Windows (*), but that is probably politically not doable on Debian.   If that is not possible, it would make sense to break the chain up and put them all in VMs of Debians of varying eras. That would decrease the maintenance for a new version at the expense of it remaining a mostly manual process.

(*) Windows is also not 100% binary stable (e.g. Windows Vista breaking due to requiring manifests for e.g. "install" binaries, but it is miles better, and more importantly you don't have to redo it for each version every 3 years.   On Windows the binutils elimination is also quite complete (internal AS and LD and even an alternative to windres in the next major version) which further reduces complications.

Thank you for your elaborated posts on my question regarding external factors that influences (or can influence) the build-process. I gladly take them into consideration in trying to figure out what goes wrong for the older builds.

Whatever the culprit, it seems to have been addressed and/or fixed with 3.20 although I am aware it could be pure coincidence (which, if the case, is what concerns me more  ).

No problem. Note that the topic is not taboo and I will help where I can with questions, but to avoid contention be clear in your reasoning, and stay clear of the wordings that introduced the confusion with Mark  (using phrases as "should work", more or less demanding support for unsupported options, based what happened to work on old debians or allowed version in the Makefile).

If you have n FPC versions that each ran on a couple debian versions from their time, you can imagine the version matrix.  And they all will have to made to run on a new version every 2-3 years. (see the VM remark above).

If I were you I would find out how debian handles bootstrapping now. (if they bootstrapped once and now keep all versions generated with that as "blessed", or have a live setup they regularly use), and read the buildfaq (linked in the doc section of the freepascal site). It is an old and rambling document where I put a lot of what I found out about building  from source , cross or otherwise.

[dbannon]

If I were you I would find out how debian handles bootstrapping now. (if they bootstrapped once and now keep all versions generated with that as "blessed", or have a live setup they regularly use), and read the buildfaq (linked in the doc section of the freepascal site)....

Debian, as a matter of policy, initially build on Sid, using Sid's current compiler. So, the latest FPC in Testing FPC.3.2.2-18, was initially built using FPC.3.2.2 (and presumably -17). The -17 and -18 refer to repackaging runs, can include application of custom patches.
 
https://buildd.debian.org/status/fetch.php?pkg=fpc&arch=amd64&ver=3.2.2%2Bdfsg-18&stamp=1673103546&raw=0

That rather surprises me because I though there was no guarantee that a FPC release can build itself ?  Not sure if that ever happens nor what Debian would then do ...

Because Sid is a rolling release (using 'release' loosely) they would never be attempting to build with anything older than the immediate predecessor.

Davo

[TRon]

@marcov:
Thank you very much again for such a extensive reply.

That gave me exactly the input I required.

Sorry to say/conclude that you probably could have have linked to this thread ergo I apologize for having wasted your time simply because I have been absent for a while (I did not notice that thread earlier nor the report in the bugtracker).

There seem to be several issues, one mentioned as in the linked thread/tracker but second in that debian itself distributed a broken glibc version. Seems to have been fixed but I'd say there are still a few things off (see also my results). My bookworm system currently runs with GLIBC 2.36-8.

Another issue seems to be related to (newly introduced ?) compressed sections. I did not fully understood the gist of it yet. I'll try to read into that later.

As already mentioned/suggested in the aforementioned thread and repeated in your post it would currently be the best solution to do as advised and maintain several systems for backwards compatibility (for as long as required). I will do that for now (maintaining several different libc versions on a single system is a bit too much for me).

Also thank you for the suggestion of having a look at your build-faq. I should be able to cite the contents of that FAQ from memory now, after having used it as a tome for all these years

I'll have renewed look when some of that pesky dust settles down a bit.

[marcov]

(the bootstrap compiler is a static binary, so shouldn't suffer from symbol versioning)

[TRon]

@marcov:
I agree that it should not matter, yet these errors persist.

Since the builds seems to suggest to bail out on the linker i've tried using other binutil versions (2.30..2.39). None of them fixed the issues (it gets even worse but that is to be expected)

But even the more strange: research seem to suggest I need even newer binutils (>2.40) to fix the issue (not a link present atm).

I tried compiling FPC versions in backwards order (e.g. bootstrap newer than sources) but that only goes sofar (funny failures though) 

Also there is the distinct mention of a missing section while linking, see also https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1023661 (bugreport mentions fixed for v2.36-5)

At least I'm not the only one :-)