Recent

Author Topic: Linker does not exclude unused procs if FPC option -g used (64-bit only!)  (Read 7511 times)

Hartmut

  • Hero Member
  • *****
  • Posts: 749
I hope I choosed a convenient subforum - if not, please let me know.

Versions:
 - Lazarus 2.0.6 with FPC 3.0.4 (on Linux installed by fpcupdeluxe with 32-bit cross-compile)
 - OS: Linux Ubuntu 18.04 (64-bit) & Windows 7 (32-bit)
 
Problem1 + Question1:
I saw that when I compile a program on Windows, that unused procedures and functions were excluded from the executable, while when compiling the same program on Linux, unused procs were not excluded from the executable. To test this, I reduced my bigger program to this:

Code: Pascal  [Select][+][-]
  1. unit unit1;
  2.  
  3. {$mode objfpc}{$H+}
  4.  
  5. interface
  6.  
  7. procedure test1;
  8. procedure test2;
  9.  
  10. implementation
  11.  
  12. procedure test1;
  13.    begin
  14.    writeln('This_is_a_test1');
  15.    end;
  16.  
  17. procedure test2;
  18.    begin
  19.    writeln('This_is_a_test2');
  20.    end;
  21.  
  22. end.
  23.  
  24.  
  25. program demo1;
  26.  
  27. {$mode objfpc}{$H+}
  28.  
  29. uses unit1;
  30.  
  31. procedure test3;
  32.    begin
  33.    writeln('This_is_a_test3');
  34.    end;
  35.  
  36. begin
  37. {$IFDEF CPU32}
  38.    writeln('32-bit');
  39. {$ELSE}
  40.    writeln('64-bit');
  41. {$ENDIF}
  42. test1;
  43. end.

For this test I used FPC directly without Lazarus:
 - on Windows => unused procs were excluded (only 32 bit tested)
 - on Linux => unused procs were not excluded (32-bit and 64-bit tested)
I tested this by searching the executable for "This_is_a_test". One match = "This_is_a_test1" means, unused procs were excluded. 3 matches = "This_is_a_test1" .. "This_is_a_test3" mean, they were not excluded.
 
After spending a lot of time I found out, that this is called "smartlinking" and that compiler options -CX and -XX are involved and that this part in fpc.cfg

#ifdef cpui8086
-CX
-XX
#endif


on Windows obviously worked, while on Linux this did not work.
After adding -CX and -XX in fpc.cfg for Linux => unused procs were excluded (32-bit and 64-bit).
Problem1 is now solved by this, but I want to ask:

Question1:
Why does "#ifdef cpui8086" work on Windows and not on Linux (on the same computer, with the same CPU)? Does anyone know a documentation about the definition of "cpui8086" (and the other possible conditionals) in fpc.cfg?

Problem2 + Question2:
First I thought, my problem was solved. But when I compiled my program via Lazarus, I saw:
 - on Windows => unused procs were excluded (only 32 bit tested)
 - on Linux (compile as 32-bit) => unused procs were excluded
 - on Linux (compile as 64-bit) => unused procs were not excluded

After spending again more time I found out, that Lazarus sets compiler options -g and -gl which mean:
  -g  = Generate debug information (default format for target)
  -gl = Use line info unit (show more info with backtraces)

And when you compile a 32-bit program with -g and/or -gl => unused procs are excluded.
And when you compile a 64-bit program with -g and/or -gl => unused procs are not excluded.

Question2:
Why are unused procs not excluded with compiler options -g and/or -gl on 64-bit, while they are excluded on 32-bit? Is this "impossible"? What must I do, that unused procs are excluded with compiler options -g and/or -gl on 64-bit too? (I'm not familiar with the linker and still a beginner on Linux).

For me it is important, that if a program crashes, that I see as much debug infos as possible. Therefore I will never compile "release versions" with reduced debug infos. But I want to exclude all unused procs (on 32-bit and on 64-bit). Is that possible and how?

I attached my little demo program and the used fpc.cfg. Thanks a lot in advance for your help. 

lucamar

  • Hero Member
  • *****
  • Posts: 4219
Question1:
Why does "#ifdef cpui8086" work on Windows and not on Linux (on the same computer, with the same CPU)? Does anyone know a documentation about the definition of "cpui8086" (and the other possible conditionals) in fpc.cfg?

That section should not be pulled for anything other than the MSDOS target since it references compiling for x86 16 bit. My guess? The smartlinking options for Windows are coming from elsewhere.

Conditionals documentation: Compiler defines during compilation
Turbo Pascal 3 CP/M - Amstrad PCW 8256 (512 KB !!!) :P
Lazarus/FPC 2.0.8/3.0.4 & 2.0.12/3.2.0 - 32/64 bits on:
(K|L|X)Ubuntu 12..18, Windows XP, 7, 10 and various DOSes.

TRon

  • Hero Member
  • *****
  • Posts: 2513
Why does "#ifdef cpui8086" work on Windows and not on Linux (on the same computer, with the same CPU)? Does anyone know a documentation about the definition of "cpui8086" (and the other possible conditionals) in fpc.cfg?
I am unable to answer the first part of your question simply because afaik there is no documentation on that particular define (other then the sources of the compiler itself).

For the second part:
https://www.freepascal.org/docs-html/prog/progap7.html
https://wiki.freepascal.org/Platform_defines

Not all of them are documented (yet) or are upcoming defines, for example the z80 processor.

Beside those defines, also programs (source-code) is allowed to contain defines ($DEFINE HELLO), and you can pass defines manually as an option to the compiler (-d).

Especially Lazarus makes heavy use of those defines in order to make things compile as they should.

If you invoke the fpc compiler from the commandline and pass -vd or -va teh compiler then you are able to see what names the compiler actually defined for compilation (or which of those are passed along from commandline or fpc configuration file.

Depending on the name of the define, a single change could trigger several other defines to change (being defined or being undefined).

Sorry, i have no answer to your smartlinking question.

Hartmut

  • Hero Member
  • *****
  • Posts: 749
Hello lucamar again, thank you for your answer and the documentation link. You are right, "cpui8086" should be for 16-bit targets only.
On Windows, I started fpc.exe with "-vc" to show the valid conditionals and "cpui8086" was not listed. So it is a miracle to me, where/why smartlinking is enabled on Windows, because if I start FPC directly (without Lazarus), where could this option come from when not from fpc.cfg?

Hello TRon, thank you for the documentation links.
But I don't understand what your infos about defines (-d) have to do with my linker problem only for 64-bit programs (for 32-bit it works).
I started FPC on Linux with -va or -vd and got a lot of output, but don't know what to find there.

The "problem" occurs (unused procs are not excluded) with this simple command line (64-bit):
   /opt/lazarus_206/fpc/bin/x86_64-linux/fpc.sh demo1.pas -g
If I switch to 32-bit, it's ok (unused procs are excluded): 
   /opt/lazarus_206/fpc/bin/x86_64-linux/fpc.sh demo1.pas -g -Pi386
So no defines (-d) are involved.

lucamar

  • Hero Member
  • *****
  • Posts: 4219
What happens if you explicitely do:
Code: Text  [Select][+][-]
  1. /opt/lazarus_206/fpc/bin/x86_64-linux/fpc.sh -B -CX -XX -g demo1.pas
and
Code: Text  [Select][+][-]
  1. /opt/lazarus_206/fpc/bin/x86_64-linux/fpc.sh -B -CX -XX -g -Pi386 demo1.pas

But ... I'm not very sure but IIRC some targets don't permit smart-linking along with debugging info, so it's one or the other for them. What I don't remember is if *nix x86_64 is one of those :-[
Turbo Pascal 3 CP/M - Amstrad PCW 8256 (512 KB !!!) :P
Lazarus/FPC 2.0.8/3.0.4 & 2.0.12/3.2.0 - 32/64 bits on:
(K|L|X)Ubuntu 12..18, Windows XP, 7, 10 and various DOSes.

TRon

  • Hero Member
  • *****
  • Posts: 2513
Hello TRon, thank you for the documentation links.
You;re welcome, even though lucamar was first because i'm such a slow typer :-)

Quote
But I don't understand what your infos about defines (-d) have to do with my linker problem only for 64-bit programs (for 32-bit it works).
Uhm... is my eyeside perhaps bad ?

Why does "#ifdef cpui8086" work on Windows and not on Linux (on the same computer, with the same CPU)? Does anyone know a documentation about the definition of "cpui8086" (and the other possible conditionals) in fpc.cfg?
Nah... i must be imagining things ;-)

Quote
I started FPC on Linux with -va or -vd and got a lot of output, but don't know what to find there.
It is about the conditionals. They get initialised at the beginning of that output, and in case the commandline options or source-code affecting them is in effect, you can see where exactly that happens in the code/handling by the compiler (and or if such define influences other defines).

I know it can sometimes be a long read, but if you wish to know exactly what is defined where and why, then you can find the answer there in the output.

Quote
So no defines (-d) are involved.
It is more a general remark on how you can approach something like this. Do you (always) know what the cat dragged in ? :-) I for sure not (and i am to lazy to delve into each and every component/units source-code in order to see what goes around in there.

@lucamar:
But ... I'm not very sure but IIRC some targets don't permit smart-linking along with debugging info, so it's one or the other for them. What I don't remember is if *nix x86_64 is one of those :-[
At least the rtl should have been compiled with smartlinking turned on, else there is no no joy to begin with (unless that has changed somwhere along the line)

lucamar

  • Hero Member
  • *****
  • Posts: 4219
At least the rtl should have been compiled with smartlinking turned on, else there is no no joy to begin with (unless that has changed somwhere along the line)

Yes, but the release RTL, etc. is compiled without debuging info. That's the crux of the matter: smart-linking and also including debug info.
Turbo Pascal 3 CP/M - Amstrad PCW 8256 (512 KB !!!) :P
Lazarus/FPC 2.0.8/3.0.4 & 2.0.12/3.2.0 - 32/64 bits on:
(K|L|X)Ubuntu 12..18, Windows XP, 7, 10 and various DOSes.

marcov

  • Administrator
  • Hero Member
  • *
  • Posts: 11452
  • FPC developer.
Why does "#ifdef cpui8086" work on Windows and not on Linux (on the same computer, with the same CPU)? Does anyone know a documentation about the definition of "cpui8086" (and the other possible conditionals) in fpc.cfg?
I am unable to answer the first part of your question simply because afaik there is no documentation on that particular define (other then the sources of the compiler itself).

For the second part:
https://www.freepascal.org/docs-html/prog/progap7.html

What is the last line of table G.2 in that link? I'll quote it for you:

Quote
CPUI8086    indicates a 16-bit x86 target (i8086)

TRon

  • Hero Member
  • *****
  • Posts: 2513
@marcov:

Uhm  :-[

Can i stil plead bad eyesight ?  My apologies to all concerned.

thx for the correction.

PascalDragon

  • Hero Member
  • *****
  • Posts: 5481
  • Compiler Developer
Question1:
Why does "#ifdef cpui8086" work on Windows and not on Linux (on the same computer, with the same CPU)? Does anyone know a documentation about the definition of "cpui8086" (and the other possible conditionals) in fpc.cfg?

As others already wrote that is to check for the i8086 processor target. Inside the configuration file you can check for any define that the compiler either defines internally (e.g. the processor or the target OS) or that was passed on the command line using -d (e.g. RELEASE and DEBUG are two examples).

Problem2 + Question2:
First I thought, my problem was solved. But when I compiled my program via Lazarus, I saw:
 - on Windows => unused procs were excluded (only 32 bit tested)
 - on Linux (compile as 32-bit) => unused procs were excluded
 - on Linux (compile as 64-bit) => unused procs were not excluded

After spending again more time I found out, that Lazarus sets compiler options -g and -gl which mean:
  -g  = Generate debug information (default format for target)
  -gl = Use line info unit (show more info with backtraces)

And when you compile a 32-bit program with -g and/or -gl => unused procs are excluded.
And when you compile a 64-bit program with -g and/or -gl => unused procs are not excluded.

Question2:
Why are unused procs not excluded with compiler options -g and/or -gl on 64-bit, while they are excluded on 32-bit? Is this "impossible"? What must I do, that unused procs are excluded with compiler options -g and/or -gl on 64-bit too? (I'm not familiar with the linker and still a beginner on Linux).

There are two points involved: first Windows uses (by default) the internal linker, while Linux always uses ld. Also 32-bit uses a different debug format (Stabs) than 64-bit (DWARF).

Would you please test the following points:
- Windows 32-bit with -glw (which will select DWARF)
- Windows 32-bit with -Xe (which will use ld instead of the internal linker)
- Windows 64-bit (both with and without -Xe)
- Linux 32-bit with -glw
- enable -CX -XX

Hartmut

  • Hero Member
  • *****
  • Posts: 749
What happens if you explicitely do:
Code: Text  [Select][+][-]
  1. /opt/lazarus_206/fpc/bin/x86_64-linux/fpc.sh -B -CX -XX -g demo1.pas
and
Code: Text  [Select][+][-]
  1. /opt/lazarus_206/fpc/bin/x86_64-linux/fpc.sh -B -CX -XX -g -Pi386 demo1.pas
I tried both, but unfortunately it made no difference: in 32-bit unused procs are excluded, in 64-bit they are not.
But now I noticed a compiler message (only in 64-bit), which I before had overlooked: "Note: DWARF debug information cannot be used with smart linking on this target, switching to static linking".

@TRon: we have misunderstood each other. You wrote "For the second part:" and then talked about defines. I had thought, you mean my "Question2". Now I know, what you meant. Sorry for confusion.

Inside the configuration file you can check for any define that the compiler either defines internally (e.g. the processor or the target OS) or that was passed on the command line using -d (e.g. RELEASE and DEBUG are two examples).
...
There are two points involved: first Windows uses (by default) the internal linker, while Linux always uses ld. Also 32-bit uses a different debug format (Stabs) than 64-bit (DWARF).
Thank you PascalDragon for that valuable additional information. I did not know this.

Would you please test the following points:
- Windows 32-bit with -glw (which will select DWARF)
- Windows 32-bit with -Xe (which will use ld instead of the internal linker)
- Windows 64-bit (both with and without -Xe)
- Linux 32-bit with -glw
- enable -CX -XX

Here are the results:

Windows 32-bit with -glw (which will select DWARF)
<path>\fpc.exe -B -CX -XX -glw demo1.pas
=> compiler writes "Note: DWARF debug information cannot be used with smart linking on this target, switching to static linking", but unused procs are excluded!

Windows 32-bit with -Xe (which will use ld instead of the internal linker)
<path>\fpc.exe -B -CX -XX -Xe demo1.pas
=> unused procs are not excluded

Windows 64-bit (both with and without -Xe)
I only have Win7 32-bit and have no 64-bit cross-compiler

Linux 32-bit with -glw
<path>/fpc.sh -B -CX -XX -glw -Pi386 demo1.pas
=> compiler writes "Note: DWARF debug information cannot be used with smart linking on this target, switching to static linking" and unused procs are not excluded

So from my understanding:
 - Windows 32-bit with internal linker excludes unused procs even with DWARF
 - Windows 32-bit with linker "ld" does not exclude unused procs
 - Linux 32-bit with DWARF does not exclude unused procs
 
Does that mean, that smartlinking with debug infos enabled is impossible on Linux 64-bit?

PascalDragon

  • Hero Member
  • *****
  • Posts: 5481
  • Compiler Developer
So from my understanding:
 - Windows 32-bit with internal linker excludes unused procs even with DWARF
 - Windows 32-bit with linker "ld" does not exclude unused procs
 - Linux 32-bit with DWARF does not exclude unused procs
 
Does that mean, that smartlinking with debug infos enabled is impossible on Linux 64-bit?

Possibly, yes. I'll have to take a look myself, maybe I'll find the time for this today evening, but I don't know yet.

Hartmut

  • Hero Member
  • *****
  • Posts: 749
Possibly, yes. I'll have to take a look myself, maybe I'll find the time for this today evening, but I don't know yet.

Take as many time as you want. Thanks a lot for your help.

TRon

  • Hero Member
  • *****
  • Posts: 2513
@TRon: we have misunderstood each other. You wrote "For the second part:" and then talked about defines. I had thought, you mean my "Question2". Now I know, what you meant. Sorry for confusion.
I apologise in return for not having articulated that enough, causing the misunderstanding.

Hartmut

  • Hero Member
  • *****
  • Posts: 749
So from my understanding:
 - Windows 32-bit with internal linker excludes unused procs even with DWARF
 - Windows 32-bit with linker "ld" does not exclude unused procs
 - Linux 32-bit with DWARF does not exclude unused procs
 
Does that mean, that smartlinking with debug infos enabled is impossible on Linux 64-bit?

Possibly, yes. I'll have to take a look myself, maybe I'll find the time for this today evening, but I don't know yet.

If you need more time, this would be ok. I only want to avoid, that you "forgot" me. In this case please allow me to send you a gentle memorization. Thanks a lot for your help.

 

TinyPortal © 2005-2018