Lazarus

Free Pascal => General => Topic started by: Avinash on January 18, 2020, 07:31:41 am

Title: Is it possible for the compiler not remove unused variables from the code?
Post by: Avinash on January 18, 2020, 07:31:41 am
I port the code from Turbo Pascal, and this feature is used: the variables are initialized in the data segment (typed consts), but then they are accessed by offsets like array, and not by identifiers. But FPС removes variables that are not explicitly accessed. Is it possible to change this behavior?
Title: Re: Is it possible for the compiler not remove unused variables from the code?
Post by: Thaddy on January 18, 2020, 08:54:29 am
What TP version ?
Title: Re: Is it possible for the compiler not remove unused variables from the code?
Post by: Avinash on January 18, 2020, 09:06:48 am
Borland Pascal 7.0
Title: Re: Is it possible for the compiler not remove unused variables from the code?
Post by: ccrause on January 18, 2020, 10:08:05 am
Can you give a small piece of code that demonstrates the problem?  Below a trivial example that seem to work on FPC 3.0.4 (x64 and x32):
Code: Pascal  [Select][+][-]
  1. program project1;
  2. const
  3.   aa: word = $BAD;
  4.   bb: word = $F00D;
  5.   cc: word = $CAFE;
  6. var
  7.   i: integer;
  8.  
  9. begin
  10.   i := 0;
  11.   repeat
  12.     writeln(i, #9, HexStr(PByte(@aa)[i], 2));
  13.     inc(i);
  14.   until i > 32;
  15. end.

Output:
Code: Text  [Select][+][-]
  1. 0       AD
  2. 1       0B
  3. 2       00
  4. 3       00
  5. 4       00
  6. 5       00
  7. 6       00
  8. 7       00
  9. 8       00
  10. 9       00
  11. 10      00
  12. 11      00
  13. 12      00
  14. 13      00
  15. 14      00
  16. 15      00
  17. 16      0D
  18. 17      F0
  19. 18      00
  20. 19      00
  21. 20      00
  22. 21      00
  23. 22      00
  24. 23      00
  25. 24      00
  26. 25      00
  27. 26      00
  28. 27      00
  29. 28      00
  30. 29      00
  31. 30      00
  32. 31      00
  33. 32      FE

One thing to note is the alignment, here the variables are aligned to 16 byte boundaries.  I would hazard a guess that the default alignment has changed from BP7.

EDIT: I noticed the alignment concern is already under discussion in a separate thread (https://forum.lazarus.freepascal.org/index.php/topic,48174.msg346548/topicseen.html#new).
Title: Re: Is it possible for the compiler not remove unused variables from the code?
Post by: Avinash on January 18, 2020, 10:35:36 am
Hm... Your example doesn’t work for me... I look at the hex-codes of exe file, there are simply no bb/cc values, but it appears if I add the lines bb:=bb; cc:=cc;
I renamed fpc.cfg (for no extra options), compile just "fpc filename.pas" — same result.

Code: Pascal  [Select][+][-]
  1. var
  2.   A, B, C, D, E, F: Integer;
  3.  
  4. begin
  5.   WriteLn( 1+
  6.            (Ofs(F) - Ofs(A)) div (Ofs(B) - Ofs(A))
  7.          );
  8. end.

FPC gets 3 (used A B F)
Borland Pascal gets 6
Win32 FPC 3.0.4
Title: Re: Is it possible for the compiler not remove unused variables from the code?
Post by: Jonas Maebe on January 18, 2020, 10:56:16 am
That is undefined behaviour. It's not FPC that removes them, but the linker. If you want to access them as an array, you must define them as an array. Disabling smart linking may be able to get you the behaviour that you want but even then it could break at any time because linkers are also allowed to move data around.
Title: Re: Is it possible for the compiler not remove unused variables from the code?
Post by: MarkMLl on January 18, 2020, 11:04:07 am
I port the code from Turbo Pascal, and this feature is used: the variables are initialized in the data segment (typed consts), but then they are accessed by offsets like array, and not by identifiers. But FPС removes variables that are not explicitly accessed. Is it possible to change this behavior?

Please could we have a reality check here. That could read that you are attempting to access items using pointers rather than indexing through the array by name, but looking later in the thread, your problem appears to be that you are attempting to use the Ofs() function.

https://www.freepascal.org/docs-html/current/rtl/system/ofs.html documents that "Ofs returns the offset of the address of a variable. This function is only supported for compatibility. In Free Pascal, it returns always the complete address of the variable, since Free Pascal is a 32/64 bit compiler." so in practice I don't know whether the code you have posted as an example will ever do what you expect on anything other than a DOS system where the segment:offset concept is valid.

MarkMLl


Title: Re: Is it possible for the compiler not remove unused variables from the code?
Post by: PascalDragon on January 18, 2020, 11:10:14 am
Hm... Your example doesn’t work for me... I look at the hex-codes of exe file, there are simply no bb/cc values, but it appears if I add the lines bb:=bb; cc:=cc;
I renamed fpc.cfg (for no extra options), compile just "fpc filename.pas" — same result.
As Jonas said you're relying on undefined behavior. What are you trying to achieve with your code? Maybe we can help you find a better solution.
Title: Re: Is it possible for the compiler not remove unused variables from the code?
Post by: Avinash on January 18, 2020, 11:51:19 am
It's not FPC that removes them, but the linker.
Indeed. When i compile as fpc -Aas or fpc -Awasm i got desired result.
But i can't compile subject sources, because these assemblers exit with errors on instructions like
Code: Pascal  [Select][+][-]
  1. movl    %cx,%edx
  2. ...
  3. addl    %dx,%edi
  4. ...
  5. movw    %esi,%dx
  6. ...
  7. cmpl    %dx,%edi

Although the default assembler build everything fine. I'll try with a Nasm may be...

What are you trying to achieve with your code? Maybe we can help you find a better solution.
That software using Turbo Vision. In this library in order to save and load objects from files (saving current state, language resources) they need to be registered using records of the form
Code: Pascal  [Select][+][-]
  1.    RDialog: TStreamRec = (
  2.      ObjType: idDialog;                               { Register id = 10 }
  3.      VmtLink: TypeOf(TDialog);
  4.      Load:  @TDialog.Load;                            { Object load method }
  5.      Store: @TDialog.Store                            { Object store method }
  6.    );

and RegisterType procedure, like (Free Vision code):

Code: Pascal  [Select][+][-]
  1. PROCEDURE RegisterDialogs;
  2. BEGIN
  3.    RegisterType(RDialog);                             { Register dialog }
  4.    RegisterType(RInputLine);                          { Register inputline }
  5.    RegisterType(RButton);                             { Register button }
  6.    RegisterType(RCluster);                            { Register cluster }
  7.    RegisterType(RRadioButtons);                       { Register radiobutton }
  8.    RegisterType(RCheckBoxes);                         { Register check boxes }
  9.    RegisterType(RMultiCheckBoxes);                    { Register multi boxes }
  10.    RegisterType(RListBox);                            { Register list box }
  11.    RegisterType(RStaticText);                         { Register static text }
  12.    RegisterType(RLabel);                              { Register label }
  13.    RegisterType(RHistory);                            { Register history }
  14.    RegisterType(RParamText);                          { Register parm text }
  15.    RegisterType(RCommandCheckBoxes);
  16.    RegisterType(RCommandIcon);
  17.    RegisterType(RCommandRadioButtons);
  18.    RegisterType(REditListBox);
  19.    RegisterType(RModalInputLine);
  20.    RegisterType(RListDlg);
  21. END;

In subject sources, unstead of calling hundred+ times of RegisterType, these records (like RDialog before) all are collected in a separate unit and are simply called in a loop:
Code: Pascal  [Select][+][-]
  1.  
  2. A: TStreamRec = (...
  3. B: TStreamRec = (...
  4. ...more than hundred...
  5. Z: TStreamRec = (...
  6.  
  7. var
  8.   P: PStreamRec;
  9.   I: Integer;
  10. begin
  11.   P := @A;
  12.   I :=Ofs(B) - Ofs(A);
  13.   repeat
  14.     RegisterType(P^);
  15.     Inc(PtrRec(P).Ofs, I);
  16.   until PtrRec(P).Ofs > Ofs(Z);
  17. end;

And because here explicitly used just three records, all the rest are missing in the binary obtained by FPC. That my current problem.
Title: Re: Is it possible for the compiler not remove unused variables from the code?
Post by: Jonas Maebe on January 18, 2020, 11:58:24 am
It's not FPC that removes them, but the linker.
Indeed. When i compile as fpc -Aas or fpc -Awasm i got desired result.
-Aas/-Awasm shows you the generated assembly code, not the linked binary. The data will always be in the assembly/object code. The linker runs afterwards and links all assembly code together. It's at that stage that unused code and data will be removed.`

You cannot do what you want to do. It will never be supported and can crash at any point. You must explicitly either put that data in an array or record, or load them one by one.
Title: Re: Is it possible for the compiler not remove unused variables from the code?
Post by: Thaddy on January 18, 2020, 12:02:38 pm
That is undefined behaviour. It's not FPC that removes them, but the linker. If you want to access them as an array, you must define them as an array. Disabling smart linking may be able to get you the behaviour that you want but even then it could break at any time because linkers are also allowed to move data around.
That may be the case, but at least for TP it proves to be well documented behavior. Not undefined. He has a definite point here and added the official TP manual entry. I checked that and tested it.
That said, indeed, an array would work around it, or a record. (packed)
Title: Re: Is it possible for the compiler not remove unused variables from the code?
Post by: Jonas Maebe on January 18, 2020, 12:09:08 pm
That is undefined behaviour. It's not FPC that removes them, but the linker. If you want to access them as an array, you must define them as an array. Disabling smart linking may be able to get you the behaviour that you want but even then it could break at any time because linkers are also allowed to move data around.
That may be the case, but at least for TP it proves to be well documented behavior. Not undefined. He has a definite point here and added the official TP manual entry. I checked that and tested it.
We cannot support emulating the TP or Delphi code generators or linkers. We are only compatible at the language level.
Title: Re: Is it possible for the compiler not remove unused variables from the code?
Post by: ccrause on January 18, 2020, 12:14:56 pm
That is undefined behaviour. It's not FPC that removes them, but the linker...
If you are using a Gnu linker then the option --undefined=symbolname can be used to stop the linker from discarding unreferenced symbols.  Give the variables public names so that you don't have to worry about mangled names:
Code: Pascal  [Select][+][-]
  1. const
  2.   a: word = $BB; public name 'a';
  3.   b: word = 13; public name 'b';

Then pass the following linker option via FPC:
Code: Text  [Select][+][-]
  1. fpc -k--undefined=a -k--undefined=b ...
Title: Re: Is it possible for the compiler not remove unused variables from the code?
Post by: Jonas Maebe on January 18, 2020, 12:35:11 pm
That is undefined behaviour. It's not FPC that removes them, but the linker...
If you are using a Gnu linker then the option --undefined=symbolname can be used to stop the linker from discarding unreferenced symbols.  Give the variables public names so that you don't have to worry about mangled names:
Code: Pascal  [Select][+][-]
  1. const
  2.   a: word = $BB; public name 'a';
  3.   b: word = 13; public name 'b';

Then pass the following linker option via FPC:
Code: Text  [Select][+][-]
  1. fpc -k--undefined=a -k--undefined=b ...

This will still not guarantee that the variables appear in the same order as in the source, even if that will usually be the case. Additionally, the OP is targeting Win32, which does not use the GNU Linker (I'm not sure we even support the GNU linker for that platform).

Again: please do not try to hack this in. In FPC, this assumption is completely unreliable and can break at any time for various reasons (not just the linker).
Title: Re: Is it possible for the compiler not remove unused variables from the code?
Post by: MarkMLl on January 18, 2020, 01:08:05 pm
Again: please do not try to hack this in. In FPC, this assumption is completely unreliable and can break at any time for various reasons (not just the linker).

Jonas, do Seg() and Ofs() generate portability warnings from the compiler? IMO they should, and that should have been sufficient to warn OP that he was on very thin ice.

Mark
Title: Re: Is it possible for the compiler not remove unused variables from the code?
Post by: PascalDragon on January 18, 2020, 01:19:43 pm
In subject sources, unstead of calling hundred+ times of RegisterType, these records (like RDialog before) all are collected in a separate unit and are simply called in a loop:
Code: Pascal  [Select][+][-]
  1.  
  2. A: TStreamRec = (...
  3. B: TStreamRec = (...
  4. ...more than hundred...
  5. Z: TStreamRec = (...
  6.  
  7. var
  8.   P: PStreamRec;
  9.   I: Integer;
  10. begin
  11.   P := @A;
  12.   I :=Ofs(B) - Ofs(A);
  13.   repeat
  14.     RegisterType(P^);
  15.     Inc(PtrRec(P).Ofs, I);
  16.   until PtrRec(P).Ofs > Ofs(Z);
  17. end;

And because here explicitly used just three records, all the rest are missing in the binary obtained by FPC. That my current problem.

You could try the following (untested):

Code: Pascal  [Select][+][-]
  1. const
  2.   MyTypes: array[0..42 { whatever your count is }] of TStreamRec = (
  3.     ( ... ),
  4.     ( ... ),
  5.     ...
  6.     ( ... )
  7.   );
  8.  
  9. var
  10.   I: Integer;
  11. begin
  12.   for I := Low(MyTypes) to High(MyTypes) do
  13.     RegisterType(MyTypes[I]);
  14. end.
Title: Re: Is it possible for the compiler not remove unused variables from the code?
Post by: PascalDragon on January 18, 2020, 01:25:33 pm
Again: please do not try to hack this in. In FPC, this assumption is completely unreliable and can break at any time for various reasons (not just the linker).

Jonas, do Seg() and Ofs() generate portability warnings from the compiler? IMO they should, and that should have been sufficient to warn OP that he was on very thin ice.
Ofs and Seg do not require any portability warnings, cause they work correctly in flat memory (segment 0 and address in that segment encompassing the whole address space).

The problem here is not that Ofs is not working, but that the layout of the binary is different from what the code expected in TP.

This will still not guarantee that the variables appear in the same order as in the source, even if that will usually be the case. Additionally, the OP is targeting Win32, which does not use the GNU Linker (I'm not sure we even support the GNU linker for that platform).
Yes, the GNU linker still works, I last tested it when implementing the BigObj COFF format. To enable it the -Xe option needs to be used.
Title: Re: Is it possible for the compiler not remove unused variables from the code?
Post by: Avinash on January 18, 2020, 03:18:01 pm
Yes, the GNU linker still works, I last tested it when implementing the BigObj COFF format. To enable it the -Xe option needs to be used.
It seems that -Xe helps (alone, without public names). Executable size increased by ~40% but this is not a price.
I got compilation with -Aas/wasm/nasm, but without the ld.exe there really is no desired result.

Again: please do not try to hack this in. In FPC, this assumption is completely unreliable and can break at any time for various reasons (not just the linker).
Maybe, but for now I’ll try with the ld.exe. If it will ultimately works, then why not...
Thanks for help.
Title: Re: Is it possible for the compiler not remove unused variables from the code?
Post by: Jonas Maebe on January 18, 2020, 03:21:44 pm
Again: please do not try to hack this in. In FPC, this assumption is completely unreliable and can break at any time for various reasons (not just the linker).
Maybe, but for now I’ll try with the ld.exe. If it will ultimately works, then why not...
[/quote]
Because, as I mentioned, it could stop working at any time. This is not how you develop reliable software. It's like keeping an uninitialised variable in your code because "it worked when I tested it".
Title: Re: Is it possible for the compiler not remove unused variables from the code?
Post by: MarkMLl on January 18, 2020, 04:48:05 pm
Maybe, but for now I’ll try with the ld.exe. If it will ultimately works, then why not...
Thanks for help.

Because Jonas is one of the core developers and has told you that's it's unwise. That is sufficient reason to look for an alternative solution.

MarkMLl
Title: Re: Is it possible for the compiler not remove unused variables from the code?
Post by: jamie on January 18, 2020, 11:51:45 pm
how about all of the needed fields inside a Record and then make a simple reference to that record so the compiler will not remove it..

 Actually the reference could be the start of the record..
Title: Re: Is it possible for the compiler not remove unused variables from the code?
Post by: PascalDragon on January 19, 2020, 10:10:14 am
We're talking about record constants of a preexisting type here. The approach I showed would be simpler in that case.
Title: Re: Is it possible for the compiler not remove unused variables from the code?
Post by: jamie on January 19, 2020, 03:43:08 pm
I remember writing DOS drivers using TP because I could remove the System unit and it would allow me to keep data at the head end for me to define to header of the driver  :D
TinyPortal © 2005-2018