[SOLVED] Incompatible PPU and other compilation problems

[avra]

This topic is a continued discussion from OPM forum thread about issues found during packages compilation. You can find that discussion here:
https://forum.lazarus.freepascal.org/index.php/topic,34297.msg333621.html#msg333621

Here is the summary of "Incompatible PPU" bug from that discussion:

I have some NewPackage that uses already compiled/installed OldPackage. When I try to install NewPackage Lazarus complains that PPU from OldPackage is not compatible. It happens first time after adding about 1/3 of the OPM packages that can be installed with current LAZ/FPC combo. Then I have to manually apply workaround to be able to continue installation of packages. Then after 5-10 packages it strikes again and the same workaround helps again. It repeats like that. PPU bug seams to prefer some packages just a little bit, but there is no rule and affected package may be random. Bug seams to strike earlier if I use -O3 instead of -O2. My "optimal" compile switches are "-O2 -g- -Xs -CpPENTIUMM -CfSSE3" but bug shows even with no optimization at all.
- Here is the mentioned workaround: https://forum.lazarus.freepascal.org/index.php/topic,43646.msg326093.html#msg326093
- Here is some relevant discussion from long time ago: https://forum.lazarus.freepascal.org/index.php/topic,43665.msg308117.html#msg308117
- Here is my old bug report: https://bugs.freepascal.org/view.php?id=34899

My latest test was on LAZ trunk and FPC trunk. Before that with LAZ trunk and FPC 3.2.0. All the same. It was also tested with official LAZ, and also with LAZ fixes and FPC fixes and the only difference is that I had to patch FPC 3.0.4 or 3.0.5 first to remove 2GB Win32 limit because that limit was hitting before PPU bug showed. All tests were run on Win10x64 and all tested LAZ and FPC were 32bit.

Good news. We got help from  a more experienced core member. Invalid ppu is just a generic messages, to find out more info, we have to compile FPC with: -vut .
Possible reasons for invalid ppu message:
- a package does not have a distinct output directory.
- packages sharing source or output directories.
- include files with same name.
- bug in FPC.

I have put -vut into FPC options of newly fpcupdeluxe made trunkgit LAZ and trunkgit FPC, as on the attached options.png. I do not know how to check if -vut was really applied, but since subjectively Lazarus compilation lasted much longer I guess that it was. If you know how can I check if options were applied please tell me. 32bit Lazarus was created and I have tested it on Win10x64.

This time, at first I didn't use any optimizations until 3rd problem showed, but let's describe 1st problem fist:

Problem1:   I started adding almost all standard Lazarus packages that could be compiled on my trunk LAZ and trunk FPC. After that I started adding OPM packages as described earlier - in groups of 1-2 starting letters. First all packages starting with A and B that could be compiled, then C and D, etc. After a while, I received fatal checksum changed error during compilation related to synhighlighterlua. I guess it was related to unit duplication, and seamed unrelated to Incompatible PPU problem. Anyway, full compilation message output is in Problem1.FileChecksum.txt file of attached ZIP archive. To continue with compilation, I simply uninstalled related package. It is maybe worth mentioning that my Incompatible PPU workaround does not solve checksum changed error.

Problem2:   After compilation of some packages, another fatal checksum changed error showed during compilation related to bgrabitmap. There were other duplicate units but I could not find that bgrabitmap was reported as duplicate, so it was not exactly the same as Problem1, but similar enough. It also seamed unrelated to Incompatible PPU problem, so I simply uninstalled bgrabitmap and dependent packages and continued compilation. Full compilation message output is in Problem2.FileChecksum.txt file of attached ZIP archive.

Problem3:   Now IDE has completely lost some packages. It wasn't the first time I faced this problem, but it didn't show for quite a while. I am not sure, but this one might be related to OPM or IDE. It does not look related to Incompatible PPU problem at all. Screenshot can be seen on attached Problem3.png. Looks like directories of many packages have simply been wiped out. For example, Synapse40.1 directory had only these 2 files:

...\synapse40.1\laz_synapse.pas
...\synapse40.1\lib\i386-win32\laz_synapse.compiled

and in ct4laz directory, folder pl_examples was missing completely, and this is the content of pl_0_libs folder:

...\ct4laz\pl_components\pl_0_libs\pl_0_libs.pas
...\ct4laz\pl_components\pl_0_libs\lib\i386-win32\pl_0_libs.compiled

or pl_cindy:

...\ct4laz\pl_components\pl_cindy\pl_cindy.pas
...\ct4laz\pl_components\pl_cindy\lib\i386-win32\win32  <<< empty dir standing there and nothing else

or pl_graphics32:

...\ct4laz\pl_components\pl_graphics32\pl_graphics32.pas
...\ct4laz\pl_components\pl_graphics32\lib\i386-win32\win32  <<< empty dir standing there and nothing else

You can also see what IDE showed after restart looking at attached 1stIdeMsgAfterReportingLostLPK.png. To solve this problem I had to uninstall lots of missing packages and dependencies, and during all that at moments I thought I could not make a working IDE in normal ways so I tried with optimized settings -O2 -g- -Xs. I do not know if that helped or something else, but after some time I got a working IDE again and could continue compilation. Full compilation message output is in Problem3.LostLPK.txt file of attached ZIP archive.

Problem4:   Finally, after a while I faced Incompatible PPU problem. Full compilation message output is in Problem4.IncompatiblePPU.txt file of attached ZIP archive. Somehow, logged messages do not show Incompatible PPU but hey show it on the screen. I wanted to attach screenshot but forum message limit was reached so if needed I can attach it in next post. Exact messages were these:

...
Compile package tponguard 1.20.1: Exit code 1, Errors 1
Fatal: Cannot find Graphics used by qonguard2, incompatible ppu=c:\prg\lazarus\trunkall\lazarus\lcl\units\i386-win32\graphics.ppu, package LCLBase
...

I have tried my workaround and it worked. Now when I think about it maybe I shouldn't - to let it be available for testing. If that will be needed then I can continue installing packages and error will show again.

I am available for all tests until September 18th when I go on a 2 weeks vacation. If needed I can 7zip whole fpcupdeluxe dir and put it to some file sharing service (600MB+). Dir just needs to be copied to C:\ and IDE can be started by shortcut that is inside of the dir, so we can say it is partially portable. It will work on your pc if you start it from the same dir as it is on my pc.

EDIT: Marked topic as solved.

[avra]

Screenshot with Incompatible PPU error.

[avra]

I have checked c:\PRG\Lazarus\TrunkAll\fpc\bin\i386-win32\fpc.cfg and at the end I can see that fpcup has appended some lines there, but there is no sign of -vut. How can I check if these compile switches have been really applied to FPC?

[balazsszekely]

@avra

Thank you again for testing. Your help is much appreciated. I will send a mail with a link to this thread. I cannot comment on Problem1 and Problem2 since I have no idea for the reason. However looking at the attached zip, more precisely to "Problem4.IncompatiblePPU.txt" it looks like there are a lot of "duplicate unit" warnings, which can cause the incompatible.ppu error(?). Problem 3 will most likely go away after 1,2 and 4 is fixed.

[BrunoK]

One of the  problems for "Duplicate file" message, is that *.lfm files are copied to the  $(TargetCPU)-$(TargetOS) directory during compilation. These files are not cleaned up when rebuilding programs (and also lazarus).

It would have been convenient if they would be renamed during that copy to the library dir to something like SomeUnit.lfu (.lfm -> .lfu) and be cleaned up before compilation.

That would simplify big clean-ups (those I do with recursive del *.o, *.ppu, *.a, *.rsj in my lazarus dir) before full rebuilds of applications or of lazarus itself.

[mattias]

Problem1:   I started adding almost all standard Lazarus packages that could be compiled on my trunk LAZ and trunk FPC. After that I started adding OPM packages as described earlier - in groups of 1-2 starting letters. First all packages starting with A and B that could be compiled, then C and D, etc. After a while, I received fatal checksum changed error during compilation related to synhighlighterlua. I guess it was related to unit duplication, and seamed unrelated to Incompatible PPU problem. Anyway, full compilation message output is in Problem1.FileChecksum.txt file of attached ZIP archive. To continue with compilation, I simply uninstalled related package. It is maybe worth mentioning that my Incompatible PPU workaround does not solve checksum changed error.

"Checksum changed" points to a serious problem in your setup and can easily lead to the "incompatible ppu" problem. A ppu is incompatible if one of its checksums changed and there are no sources to recompile the unit.
When a package has been compiled, its ppu(s) must be finished. Any recompile of this ppu or any required ppus will lead to the incompatible ppu error.

"Unit duplication" is asking for trouble. Don't.

[mattias]

What packages do you install?

[avra]

@GetMem:
I have news for Problem 3. I have continued adding packages and at one moment I got checksum changed error. After that I did CleanUp packages in OPM. Immediately after that I got message that many LPKs were lost and missing. I checked and directories were empty with just a single pas file as described earlier. So it is repeatable. In that session I have tried to install 7-8 new packages, but not only their directories were empty when error showed up, but also some older packages that I am 100% positive they were installed 2 days before that.

I have also found a package which prevents Lazarus from creating Lazarus executable on IDE rebuilding. It is pl_OpenWire, and problem was found on 32bit LAZ trunk and FPC trunk on Win10x64. It would be nice if someone else could confirm this. I did not have time for further investigation, since I am preparing for my vacation.

"Checksum changed" points to a serious problem in your setup and can easily lead to the "incompatible ppu" problem. A ppu is incompatible if one of its checksums changed and there are no sources to recompile the unit.
When a package has been compiled, its ppu(s) must be finished. Any recompile of this ppu or any required ppus will lead to the incompatible ppu error.

"Unit duplication" is asking for trouble. Don't.

I am not creating any duplicated units. I am just adding packages with OPM. I have earlier informed few authors about duplicated units they create when they have several packages and use different directories for the same output files used in several packages, and even made some patches. However it seams there are more deeply buried. I noticed them but thought that it might not be that important since last time I looked (not recently) source files were identical and I thought those reported duplicates could be safely ignored.

I have now checked a little, and latest duplication found is related to completely different synhighlighterlua.pas from ExtraHighlighters and lainzcodestudio packages. Authors should be notified.

@GetMem:
Maybe some simple certification of automated checking for duplicates before adding a package to OPM could help in such a case?

What packages do you install?

After fresh fpcupdeluxe installation I add most of standard Lazarus packages, then start adding OPM packages in groups. I then add all packages starting with letters A and B, compile them and remove ones that can not compile, then rebuild IDE and continue adding other OPM packages. I did not take notes what exact package from the group produced error. I will try to note that down next time. What I did notice is that checksum changed error seams to be consistent and always show on same package (look at attachments), while incompatible ppu error is pretty random, and usually shows for a package that was installed not in last package installation session but much, much earlier, and now that old package stops working when new packages were added, causing even some dependent packages to fail also. After incompatible ppu error I apply my workaround and I can continue adding packages. Workaround is to rebuild IDE with just "Clean common files" and "Switch after building to automatically" checked. For some reason "Clean all" does not help.

I will not be able to do any more testing until I return from my vacation. Sorry for such a bummer. I will try to make it up when I return.

Thanks everyone looking into this issue.

[balazsszekely]

@avra

Have a nice vacation.  Unfortunately I still don't know what to do, other then remove/fix(notify the maintainer) the packages with duplicate units, but I will keep trying.

PS: One more question before you go: do you rebuild the IDE after installing a set of packages?

[avra]

Have a nice vacation. 

Thanks! 

Unfortunately I still don't know what to do, other then remove/fix(notify the maintainer) the packages with duplicate units, but I will keep trying.

I understand that. What I think is that currently only someone who installs tons of packages can notice duplicates and report them, and that is far from optimal. Maybe for the future there should be some semi or fully automated way to check for duplicates before accepting new package into OPM? Semi automation could be probably achieved with currently available tools that could scan official or trunk fpc/laz dir, scan all OPM packages dir, and scan new OPM package candidate dir and report duplicates. That process could even be automated with some scripts. Full automation would need a custom solution. Maybe there is a better way, but I do not see it at the moment.

do you rebuild the IDE after installing a set of packages?

Yes. After group installation of packages, I first check if some package from the group can not be compiled and remove it. Then if all packages can be compiled I rebuild IDE. Always.

[avra]

Problem solved! 

I have applied PascalDragon's fix for sparta generics used by embedded form designer and some other packages, and since then I could install tons of components without a problem. Of course, I could not install packages not compatible with FPC 3.2.1 and Lazarus 2.11 which I used for testing - but these are not that many. I have installed most of the Lazarus original packages, almost all OPM packages, and some other packages via FPCUPDELUXE package installation method. Something I could not do previously for quite a long time.

Thanks PascalDragon!

p.s.: When using FPCUPDELUXE package installation method, since compilation switches were different between it and my optimized IDE, I could get a PPU problem. Unlike described PPU problem which could not be solved until now, this PPU problem is normal one and can be fixed by clean rebuild of the IDE. That is fine and it should work that way. I am really happy now!

EDIT: Here is the requested link to PascalDragon's fix: https://forum.lazarus.freepascal.org/index.php/topic,50528.msg368954.html#msg368954