Recent

Author Topic: [SOLVED] Poor optimization of constant folding  (Read 8592 times)

dsiders

  • Hero Member
  • *****
  • Posts: 1280
Re: Poor optimization of constant folding
« Reply #15 on: September 29, 2023, 08:26:59 am »
The full source package indeed takes around that long even with a powerful CPU, a lot of fast RAM and NVMe SSD.
I've wondered for quite some time how long it takes to do a full build of Windows 10 or 11.  That's 25 million+ lines of code, even on a really fast machine, it must be a rather "noticeable" amount of time.

I saw this some time ago:

> How long does it take to compile Windows?
>> Ales Holecek, vice president for development in Windows team said that it takes about 16 hours to build Windows 10.
>> And that it's built automatically every day during the night.
>> Oct 2008
Preview the next Lazarus documentation release at: https://dsiders.gitlab.io/lazdocsnext

marcov

  • Administrator
  • Hero Member
  • *
  • Posts: 11919
  • FPC developer.
Re: Poor optimization of constant folding
« Reply #16 on: September 29, 2023, 09:04:17 am »
Compiling FPC with FPC takes a minute or so. Compiling GCC with GCC takes around 6 hours.

IIRC Sub minute for the whole FPC repo on a Ryzen 5700X on linux. A few seconds over a minute for windows.

But that includes RTL, packages and textmode IDE

p.s. my buildscript does skip the WPO optimization cycle.
« Last Edit: September 29, 2023, 10:41:24 am by marcov »

440bx

  • Hero Member
  • *****
  • Posts: 4726
Re: Poor optimization of constant folding
« Reply #17 on: September 29, 2023, 10:29:57 am »
>> Ales Holecek, vice president for development in Windows team said that it takes about 16 hours to build Windows 10.
>> And that it's built automatically every day during the night.
>> Oct 2008
Thanks dsiders. 

16 hours... what a drag!  I'm guessing that includes all the utilities, e.g, explorer.exe, IE/Edge, some of .net, etc, along with the O/S proper.  IOW, everything that goes into "c:\windows" and possibly a few things that go into "c:\program files".

I guess he didn't mention the hardware specs. 
(FPC v3.0.4 and Lazarus 1.8.2) or (FPC v3.2.2 and Lazarus v3.2) on Windows 7 SP1 64bit.

dsiders

  • Hero Member
  • *****
  • Posts: 1280
Re: Poor optimization of constant folding
« Reply #18 on: September 29, 2023, 05:02:13 pm »
>> Ales Holecek, vice president for development in Windows team said that it takes about 16 hours to build Windows 10.
>> And that it's built automatically every day during the night.
>> Oct 2008
Thanks dsiders. 

16 hours... what a drag!  I'm guessing that includes all the utilities, e.g, explorer.exe, IE/Edge, some of .net, etc, along with the O/S proper.  IOW, everything that goes into "c:\windows" and possibly a few things that go into "c:\program files".

I guess he didn't mention the hardware specs.

That was the entire response. It's also the *only* response I've seen from a M$ employee.
Preview the next Lazarus documentation release at: https://dsiders.gitlab.io/lazdocsnext

Warfley

  • Hero Member
  • *****
  • Posts: 1733
Re: Poor optimization of constant folding
« Reply #19 on: September 29, 2023, 06:02:19 pm »
C and C++ is very slow to compile full projects, which is why buildsystems like make only compile changed files and include the linker.

For example when I last worked on a large C++ project, compiling all the project including the whole dependency chains (which included building LLVM) would take 30 minutes on my PC with 32 gigs of RAM and a 1. gen threadripper. On my Macbook air at the time it would take multiple hours.
But once compiled only changed files need to be re-compiled, so as long as you don't touch the headers, everything compiles in a minute or so.

When people talk about the build times like MS with 16 hours, those are the nightly clean builds. Of course during development you will just compile deltas, and probably also always work on smaller tests rather than always building the full system.

That said, I know of some game companies that write games in C++, which always compile during their lunch breaks, to be able to test all the new features they built in the morning then later in the afternoon.

Some of this long compilation time is due to the C language design, e.g. the usage of header files means as soon as you change one byte of a header, you must recompile all files that include that header, while Pascal with it's unit and ppu system is much better optimized (I think since C++ got modules this should work similarly, but haven't used C++ for a while now). Thats why pretty much no other language uses the include file system from C but most go with a namespace/module system like Pascal Units.

But a very big factor is, as I said previously, simply that the C and C++ compilers do more stuff under the hood.
So yes, C compilers are very slow, but a big reason for this is, because they do a lot more things. Not all of those things the C compiler does make sense in all situations, e.g. I would argue that wasting whole hours of your day due to waiting compilation may not be worth some additional optimizations you get.
C is made for kernel development and C++ is used for things like browsers and low level applications. There you need all bit of optimization you can get. But honestly, I'm not sure if that is really worth it for pascal.

Many optimization problems are NP complete or even in non NP exp-time (well optimization as a concept in general are not even computable in the first place, only a few narrowly defined specific optimization patterns can be implemented), so there is no way to make them very fast. So not having them may be the better option when you aren't building time critical software
« Last Edit: September 29, 2023, 06:08:31 pm by Warfley »

440bx

  • Hero Member
  • *****
  • Posts: 4726
Re: Poor optimization of constant folding
« Reply #20 on: September 29, 2023, 06:55:50 pm »
At least for me, compilation speed changes the way I program.

With Delphi and FPC, I do a full build every few lines of code, sometimes as few as 1 or 2 lines, build, place a breakpoint on the new code and execute under the debugger.  By the time I'm done, the amount of testing left to do is often relatively small.  This is because, even a 75,000 line program takes less than half a second to build (and, by today's standards, my machine is slow - 2.8Ghz, 10+ year old quad core)

When using C, it's a different ballgame, compiling 75,000 lines takes more than 5 seconds and that's already enough to break the smooth flow of edit/build/test.  Even a make in C usually takes longer than a full build of the same (ported) project in FPC/Delphi.

One thing that should be stated is that, during development, it really doesn't make much sense to build with deep optimizations since that gets in the way of debugging (a debugger is an excellent testing/verification tool.)
(FPC v3.0.4 and Lazarus 1.8.2) or (FPC v3.2.2 and Lazarus v3.2) on Windows 7 SP1 64bit.

Martin_fr

  • Administrator
  • Hero Member
  • *
  • Posts: 10542
  • Debugger - SynEdit - and more
    • wiki
Re: Poor optimization of constant folding
« Reply #21 on: October 04, 2023, 01:54:09 pm »

runewalsh

  • Jr. Member
  • **
  • Posts: 85
Re: Poor optimization of constant folding
« Reply #22 on: November 12, 2023, 10:38:36 pm »
Until the fix in mid-2030s

I’m a pessimist on purpose so bug fixes can be pleasant surprises.

 

TinyPortal © 2005-2018