lazbuild vs IDE: PPU checksum mismatch / loop compilation error when installing

[dmitryb]

Hi everyone,

Lazarus 4.2 (rev lazarus_4_2) FPC 3.2.2 x86_64-win64-win32/win64

I am facing a weird issue when trying to build and install our component package (ehlib.lpk) without exposing the source .pas files (providing only precompiled .ppu and .o files in the lib directory).

The Problem:
If I compile the package twice inside the Lazarus IDE (via "Clean and Compile"), it compiles successfully and can be installed into the IDE without issues.

However, if I try to automate this build process using lazbuild via a command-line script:

DOS
C:\lazarus\lazbuild.exe --build-all ehlib.lpk || Pause

REM Second build to stabilize FPC checksums
C:\lazarus\lazbuild.exe ehlib.lpk || Pause
The compilation succeeds in the console, but as soon as I start the Lazarus IDE, select the package, and click "Use" -> "Install", the IDE rebuild process fails with the following errors:

Plaintext
Build IDE: Exit code 2, Errors: 1, Warnings: 2
Warning: Version control client not in path.
Warning: Recompiling DBGridEhFindDlgs, checksum changed for C:\lazarus\EhLib\Lib\x86_64-win64\DBGridEh.ppu
DBGridEhFindDlgs.pas(86,12) Error: Can't find unit DBGridEhFindDlgs used by DBGridEh
Additional Details:
The package .lpk is configured to use the -Ur flag (Create Release Unit) in the custom options to freeze checksums:

XML
<CustomOptions Value="-Ur"/>
In ehlib.compiled, I can clearly see that -Ur is indeed passed to the compiler during the lazbuild run:

XML
<Params Value="... -FUC:\lazarus\EhLib\Lib\x86_64-win64\ -dLCL -dLCLwin32 -Ur ehlib.pas"/>
There seems to be a cyclic dependency or an inline compilation order issue between DBGridEh and DBGridEhFindDlgs.

The IDE is able to resolve this and stabilize the checksums after a couple of passes, but lazbuild fails to achieve the same state, even when run twice.

My question is: How does the IDE's compilation process differ from running lazbuild.exe in this case? What is the correct way to invoke lazbuild so that it matches the IDE's exact compilation pass and properly seals the PPU checksums for a "PPU-only" installation?

Thanks in advance for any insights!

[Martin_fr]

First of all that sounds like a known FPC bug. "Known" as in "it has be observed". Not sure as in "known where to fix". That said, in 3.3.1 the entire part that handles this has been reworked, so chances are, if/when a FPC 4.0 comes then that issues wont be there anymore.

As for what happens....
(Or my understanding of it / not confirmed by any compiler dev)

It is usually triggered by a combination of things.
- Almost (or Always?) a circular unit reference  (in your ehlib package)
- often inlining, but maybe other optimizations
- could be an "inline" that is not in the interface section, but the implementation section
- May be something else, not an inline...


When the package is compiled normally, the compiler start loading pas, and creating ppu for them.

Each ppu contains checksums (several), of any other ppu it relies on (so that if any changes, all dependents can be found and recompiled).

With circular refs, the complier must
1- load unitA.pas, read the "uses"
2- load used unitB.pas, compile the interface block
3- in UnitB implementation sees "uses UnitA"
4- go back to UnitA compile the interface part (now knowing UnitB)
    not sure, maybe compiles all of UnitA at that time)
5- Now that UnitA interface is compiled continue compile UnitB

During that it has partial ppu files. And one of them is saved with the checksum of the incomplete other. (i.e. the wrong checksum is kept, and makes it into the result).

Then when you compile, with only ppu visible => you get the "checksum changed", because it does not match.
Not finding the source is then an obvious consequence, of only seeing ppu...


2 ways of fixing tihs
a) Compile with -Ur  release units, suppress the checksum checks.
b) Compile twice (without cleaning in between)

When you compile the 2nd time, the compiler finds the existing ppu. It loads them instead of compiling the pas sources (IIRC, if the timestamp is ok). So it does not compile again. Except when it finds the one that has the wrong checksum (in the "I depend on foo@check1 => but foo itself says "@check2") then it does recompile that one file. That fixes the checksum.

It may have to recompile both, but if it does, it still had the correct checksum from the existing ppu... Basically when it has the ppu to start, then it doesn't run into the "I keep the wrong ppu" issue

[dmitryb]

Hello Martin

Dual compilation only helps when you compile the lpk file through the Lazarus IDE.
But when using the command line, dual compilation doesn't help:

Code: Pascal  [Select][+][-]

1. lazbuild --build-all ehlib.lpk
2. lazbuild ehlib.lpk
3.  

The second build doesn't recompile any units at all.

Are there any lazbuild compilation options that would fix this problem?

[Martin_fr]

Are there any lazbuild compilation options that would fix this problem?

Sorry, I don't know. I never tried with lazbuild.
Even in the IDE it is not a guarantee, just a sometimes it helps.

As indicated you should get usable ppu with
  -Ur
as option.

That generates ppu flagged as release, they afaik don't complain about the checksum. 
I have not tested, but they then also would not complain if the RTL changed. Well for diff FPC releases they would have the wrong ppu version, but for diff fixes branch builds... I don't know.