Recent

Author Topic: Optimization Level O1  (Read 893 times)

lainz

  • Hero Member
  • *****
  • Posts: 4468
    • https://lainz.github.io/
Optimization Level O1
« on: December 12, 2022, 03:21:19 pm »
Hi, there's a list of optimizations that are enabled in O1?

Because I'm working with a third party plugin called kbmMemTable, and there's a bug in a part of the code only if I have the Normal IDE compiled, when I have the Debug IDE compiled it goes away. The only difference I see is that the Normal IDE has optimizations of level 1.

Any ideas on how to fix a bug caused by optimizations?

Martin_fr

  • Administrator
  • Hero Member
  • *
  • Posts: 9864
  • Debugger - SynEdit - and more
    • wiki
Re: Optimization Level O1
« Reply #1 on: December 12, 2022, 03:38:32 pm »
There may be a list somewhere, but I don't know where....

Are you using FPC 3.2.2?
If so, have you build the IDE with -O1 (or less) ?

The FPC 3.2.2 has a known bug in the optimizer. And while it is insanely hard to trigger, it does affect FpDebug (random hangs/crashes).

It could be that this bug also happens in the code you use....
Or if you only have troubles while debugging, then maybe it's the debugger and not your app...

Also, IIRC there may be more than the one issue in 3.2.2. But I don't recall details. Though any of those should be fixed in 3.2.3.


As for the issues that I mentioned first, IIRC it is a combination of 2 optimizations (one of which is part of the peephole optimizer).

lainz

  • Hero Member
  • *****
  • Posts: 4468
    • https://lainz.github.io/
Re: Optimization Level O1
« Reply #2 on: December 12, 2022, 03:44:14 pm »
Ok thanks for your answer.

More strangely is that for testing I compiled the IDE in 'Optimized IDE' and now the bug fully goes away in Windows 32 and Linux 32 bit.

Maybe we need to upgrade to master in order to get the bug fixed independently of the optimization level?

Bogen85

  • Hero Member
  • *****
  • Posts: 595
Re: Optimization Level O1
« Reply #3 on: December 12, 2022, 03:52:27 pm »
Hi, there's a list of optimizations that are enabled in O1?

Because I'm working with a third party plugin called kbmMemTable, and there's a bug in a part of the code only if I have the Normal IDE compiled, when I have the Debug IDE compiled it goes away. The only difference I see is that the Normal IDE has optimizations of level 1.

Any ideas on how to fix a bug caused by optimizations?

https://www.freepascal.org/docs-html/prog/progse49.html
fpc -i
fpci -io

From my experience bugs that surface when optimization is enabled are usually not the fault of the of the optimizer, but the fault of the code having dubious or ambiguous logic that the optimizer exposes. When warnings and hints are enable and considered errors, when one has aggressive optimization more is often found.

I have to be a lot more selective on errors and warning when compling laz vs fpc units (I have to disable a lot of warnings for Laz units that I don't need to for FPC units but that is a separate issue).

I have ran into some actual optimizer bugs, and I've filed bug reports on those. They were not ones turned on by O1, they had to do with register variables and dead code elimination.

That being said, when someone develops code and does not have aggressive optimization enabled at the beginning of the development, the longer that code is developed the more chances it has for bugs to be present that only surface when when higher optimization levels are enabled.

Then the optimizer is blamed, when it usually (in my experience) is the code that is at fault...




lainz

  • Hero Member
  • *****
  • Posts: 4468
    • https://lainz.github.io/
Re: Optimization Level O1
« Reply #4 on: December 12, 2022, 04:15:12 pm »
I don't get what you say, I'm saying that turning on O3 fixes the issue. So higher optimization fixed it. We test our program in debug mode and release mode, so O1 and O3.

But the problem has to do with the LCL or other package and the kbmMemTable package. When compiled with O1 in a normal IDE fails. Again Optimized IDE with O3 solved the bug.

The bug isn't in our own code, but in the combination of LCL or other package and kbmMemTable, the same code with bufdataset was working fine, except that slow as hell.

Martin_fr

  • Administrator
  • Hero Member
  • *
  • Posts: 9864
  • Debugger - SynEdit - and more
    • wiki
Re: Optimization Level O1
« Reply #5 on: December 12, 2022, 04:48:56 pm »
More strangely is that for testing I compiled the IDE in 'Optimized IDE' and now the bug fully goes away in Windows 32 and Linux 32 bit.
I didn't mention, but at least the bug affection fpDebug too, is a 64bit issue (afaik / at least it depends on cpu registers, and they differ by bitness).

In any case, optimizer bugs are notoriously elusive.... One tiny change: a line of code, just re-ordering, or a compiler setting...,  and they can come and go.... They also could be present, but depend on program flow, i.e. only if certain other code was/is run before/after the bug, then they will cause an error, and otherwise not.

When I encountered the issue the first time, it took me (combined / I did have to have several goes) 4 or 5 weeks to trace it. And "trace it" does not mean to find the actual wrong code in the compiler, it meant to find the code in my app, that was incorrectly compiled, and to be able to see what had gone wrong in the generated asm. => From that info a member of the FPC team was then able to find the bug in fpc....

Quote
Maybe we need to upgrade to master in order to get the bug fixed independently of the optimization level?

FPC 3.2.3 (fixes branch) should be fine.

Fpc 3.2.2 with -O1 also should be (at least from all that I have seen so far)

Quote
The bug isn't in our own code, but in the combination of LCL or other package and kbmMemTable, the same code with bufdataset was working fine, except that slow as hell.
Probably... But never say never.

While we know there are issues in FPC, we don't know if it is them that cause this particular issue.

It is very possible, and has happen more than often enough that a bug in the "user code" (including LCL) did only led to errors when compiled with optimization. Things like missing initializations, incorrect nil checks, incorrect bool/math logic in expressions and comparisons, .... all of those may sometimes still yield correct results, unless some optimization comes in too.

And, even if the feature that breaks is in the LCL, it can still be user code. A dangling pointer write access can destroy any data of the app, any data completely unrelated to the buggy code. So in such case, the LCL could easily crash in reaction to having its data trashed.

More so, going back to the possible optimizer bug: It is possible that the optimizer incorrectly translates some of your valid code, and by that introduce a dangling pointer (or similar) issue, which in turn manifests in the LCL.


So, O3 may work... well at least today. It may fail, if you add or remove a line of code somewhere.

If indeed it is an optimizer bug, then you have 2 options:
- use -O1 (for your code, and all packages including LCL...)
- upgrade fpc to 3.2.3

If you do either of the above, and the issue goes away....
Well you still want to make sure, that you actually fixed the right thing. Because as I said "notoriously elusive.... One tiny change" => There still is the possibility that there is a bug in your code affecting the LCL (or maybe it is in the LCL....), and that changing the compiler version was just One tiny change. And the next tiny change brings it back. Or it is even still there, but just needs other input, other order of buttons pressed by user.... And so it may work for you, but the same compiled exe working for today's use-case will fail next week.

So throw in all the checks the compiler has (in different combinations).
  -CRriot
  -gh 
  -gt (with 1,2,3 or 4 "t", so you need 4 different test builds, because -gt -gt is equal to -gtt -- each compiled exe can only be tested with one of the 4 options),
  -gh (and environment HEAPTRC="keepreleased",
  add assertions, wherever you can (and run with and without them: -Sa  on or off)

And last not least, if you are on linux: use -gv and test with
   valgrind --tool=memcheck
And if you are using threads, valgrind has some thread checker too (actually several) / google that.


korba812

  • Sr. Member
  • ****
  • Posts: 394
Re: Optimization Level O1
« Reply #6 on: December 12, 2022, 05:03:00 pm »
Are you sure the problem comes from optimization? Debug IDE additionally uses "-gh" (use heaptrace unit) and "-gt" (trash local variables) parameters. Maybe they have the effect of hiding the error and that's why it works with "Debug IDE".

lainz

  • Hero Member
  • *****
  • Posts: 4468
    • https://lainz.github.io/
Re: Optimization Level O1
« Reply #7 on: December 12, 2022, 05:06:30 pm »
Hi again, last thing I've touched was kbmMemTable package to O3. Despite the IDE optimization level, and so default packages, works.

So the problem is like Martin_fr said, with O1.

Martin_fr

  • Administrator
  • Hero Member
  • *
  • Posts: 9864
  • Debugger - SynEdit - and more
    • wiki
Re: Optimization Level O1
« Reply #8 on: December 12, 2022, 05:15:00 pm »
Hi again, last thing I've touched was kbmMemTable package to O3. Despite the IDE optimization level, and so default packages, works.

So the problem is like Martin_fr said, with O1.
Actually, I was saying that - if it is an issue in the optimizer itself - it should be fixed by using -O1.
And therefore that in that case -O2 and -O3 are causing it.

However, I also said, that any change (in code or settings) can hide (not fix) the issue.

For example a dangling pointer in your code (either due to your fault, or wrongly compiled due to a compiler bug) could affect kbmMemTable, but when you change the settings O1 vs O3 for kbmMemTable, then you change the code (and mem layout) of kbmMemTable. Thus the dangling pointer error no longer hits kbmMemTable.
The dangling pointer could now hit some other code/mem. Or it could just hit unused memory, causing no noticeable effect at all.

jamie

  • Hero Member
  • *****
  • Posts: 6130
Re: Optimization Level O1
« Reply #9 on: December 12, 2022, 05:23:09 pm »
  I've seen too much code that assumes variables are initialized or just don't think about.

  I also see dead people, so pay no attention! :o

 
The only true wisdom is knowing you know nothing

Martin_fr

  • Administrator
  • Hero Member
  • *
  • Posts: 9864
  • Debugger - SynEdit - and more
    • wiki
Re: Optimization Level O1
« Reply #10 on: December 12, 2022, 05:54:02 pm »
  I also see dead people, so pay no attention! :o
Don't worry... Normal, at least if you are at an open casket...

You didn't initialize the environment, under which to evaluate your quoted statement. ;)

Fred vS

  • Hero Member
  • *****
  • Posts: 3168
    • StrumPract is the musicians best friend
Re: Optimization Level O1
« Reply #11 on: December 12, 2022, 07:25:00 pm »
Hello.

Maybe related (or not)...

What mode are you using?
If using {$mode objfpc} or {$mode delphi} fpc is more tolerant with pointer of pointer than {$mode fpc}.

For example, using fpc-llvm and any optimization makes (in the past) MSEgui applications crash.
The problem was in MSEgui code and some not dereferenced pointer and tanks to Jonas, it is fixed now.

Note that with the "pure" fpc and using optimization + {$mode objfpc}, there was no crash, seems that fpc dereferenced those pointers automatically.

So, to resume, if it is possible to use only {$mode fpc}. (who is much strict for dereferencies) it should have show at compilation the error.

And maybe using -O1 makes fpc less tolerant.

Maybe.

Fre;D
I use Lazarus 2.2.0 32/64 and FPC 3.2.2 32/64 on Debian 11 64 bit, Windows 10, Windows 7 32/64, Windows XP 32,  FreeBSD 64.
Widgetset: fpGUI, MSEgui, Win32, GTK2, Qt.

https://github.com/fredvs
https://gitlab.com/fredvs
https://codeberg.org/fredvs

lainz

  • Hero Member
  • *****
  • Posts: 4468
    • https://lainz.github.io/
Re: Optimization Level O1
« Reply #12 on: December 12, 2022, 10:31:04 pm »
Actually O3 fixed 2 issues. Strangely happening only on Linux 32.

 

TinyPortal © 2005-2018