Lazarus IDE compiling is too slow for me!!

[vfclists]

I use Lazarus for my personal needs ie as my personal program and the compile edit cycle is to slow for me.

I realized that some years ago and I quit Lazarus for my personal regular stop.

I put it down to the fact that the PC was not fast enough, ie compiling and linking and writing out to disk was too slow for my needs.

But having upgraded my hardware it is still too slow for my liking.
I'm currently using an 8th gen i5 laptop, with 16G of RAM and an Nvme2 drive and it is still not fast enough. I understand that laptops are usually throttled to avoid overheating, but once the program is compiled performance looks fast enough. Perhaps a well specced late model Ryzen or Xeon workstation I can overclock to the highest speed may be better, but I can't take it around with me. Perhaps I should switch to cloud based development.

I remember the times when in the early days of Delphi a program would be fully compiled and linked and ready to run literally before I took my finger of the keyboard. This is not a "good old days" comment. I am certain it was actually so unless my memory is befuddled.

There are two questions I want to ask -

1. How can the compile be profiled to see where the hotspots are, even if the compilation occurs in parallel?

2. After point 1 is sorted what kind of workstation or laptop would I go for? The latest Macs are out. It has to be an Intel or AMD. Laptops are preferred for portability.

I need to go from designing in the IDE to running program in less then 8 secs. Short enough to stop my attention from moving onto something else. Start up time must be as fast as it would be with an interpreter.

The LCL is the main selling point for me with Lazarus which is why I'm exploring Python, Lua and Lisp with the LCL.

Are you guys aware that some enterprising guys have both Nim and Go working with the LCL?

[TRon]

I'm currently using an 8th gen i5 laptop, with 16G of RAM and an Nvme2 drive and it is still not fast enough.

In case the latter there must be something wrong configured. nvme drives are blazingly fast but if you connect it to pcie x 1 then it will not be faster than sata.

In case the former... not using a ram drive ? It can't get any faster than that.

[Curt Carpenter]

Could you provide some actual times and file sizes?  (I'm only curious.  I use a pretty old 32bit machine on Linux and it has always seemed very fast with Lazarus.  Even rebuilding after adding a new package  only takes a minute or two.)   

[vfclists]

I'm currently using an 8th gen i5 laptop, with 16G of RAM and an Nvme2 drive and it is still not fast enough.

In case the latter there must be something wrong configured. nvme drives are blazingly fast but if you connect it to pcie x 1 then it will not be faster than sata.

In case the former... not using a ram drive ? It can't get any faster than that.

Linux caches drive writes so I've come to the conclusion that a faster drive won't help. The process must be compute or IO bound. I would think that generation 80Mb of exe and debug information would be slow, but not on an NVMe drive.

The NVMe drive is fast alright, unless the write process makes many small writes in the case of this compilation.

[vfclists]

Could you provide some actual times and file sizes?  (I'm only curious.  I use a pretty old 32bit machine on Linux and it has always seemed very fast with Lazarus.  Even rebuilding after adding a new package  only takes a minute or two.)

I think there is a dialog which provides this information but I can't remember what it is and it may not be present in the menus.

Do you know what it is or whether it requires an additional package to be installed?

[Bart]

On my old win10 laptop (i5 7th gen, 8GB RAM, SSD hardisk) building Lazarus clean takes < 2 minutes.

Bart

[vfclists]

On my old win10 laptop (i5 7th gen, 8GB RAM, SSD hardisk) building Lazarus clean takes < 2 minutes.

Bart

Mine does this in less time.

[Martin_fr]

OK, first I though the subject was about recompiling the IDE itself.

But you are talking about your app.

1) What OS?
2) Size of your project (number of units, average amount of lines per unit)
3) Size of the compiled exe (with debug info)
4) Do you have your own packages, do you edit files in those packages?

to running program in less then 8 secs.

For reference, I am using a debug build of the IDE and FPC (tiny bit slower) and an new GUI project, takes 4 seconds from pressing F9 to run.
Another project takes 7 seconds if it has a few units to rebuild (I made changes to them). So that is very close to the 8 secs you want. It may go over that time...
(I7 8700K  but I think compared to your CPU that wont make a big diff, the speed of your ssd matters / memory you have enough.)

I have a normal Lazarus build, but that sits on a HDD, and takes 10 seconds for an new empty app. (so disk speed does matter).

If you have a big project, then the extra time needed depends.
Normally only units that were changed (or depend on those) are recompiled. So not all of your units will be recompiled each time.

You should have a look at the messages window:
1) How long does it take between pressing F9 and the green line appearing "Compile project .... Success ..."
2) During this time, how long do you see the message "Linking..."
3) How much time after that, till you app runs?


If the time happens in 1 or 2 then it depends how many units needed to be recompiled.

If 1) Do you have any mega big units? (10000 lines and more)?

If it is 2 then check if you have "Smart linking" off, you do not need that for your testing and debugging. And for larger apps it can be noticeable.
You could also remove debug info from packages like the lcl.

If it is 3 (or even if not) check you debugger backend (Menu Tools > Options > Debugger Backend).
Since you have an Intel CPU you should be using FpDebug.
If you are using GDB, well yes that takes time to start your app.

units do NOT compile in parallel, unless they are in different packages.

But I wouldn't worry about that, unless you got more than a dozen units that need to be compiled for each run (because they all were changed, or are using a unit that were changed / but if the are using a unit that was changed, they all need to wait for that one)


You can rebuild both fpc and Lazarus with -O4 => but it will probably give between 3 and 5% only. (Assuming you already have them at -O2 which is default)

The IDE may be prebuild with -gh heaptrc (not sure), you could rebuild it with -gh- => might help another little bit (if it was with gh).

[Martin_fr]

Could you provide some actual times and file sizes?  (I'm only curious.  I use a pretty old 32bit machine on Linux and it has always seemed very fast with Lazarus.  Even rebuilding after adding a new package  only takes a minute or two.)

I think there is a dialog which provides this information but I can't remember what it is and it may not be present in the menus.

Do you know what it is or whether it requires an additional package to be installed?

That dialog has long been abandoned.

[vfclists]

Does anyone have some experience profiling Lazarus/FPC with the kind of tools that can generate a flamegraph?

I don't know whether Lazarus has to be built with support for profiling, or it is the tool themselves that run the projects make commands and analyze their performance.

After you have done your editing and are ready to compile, what is the command actually run to build the application? I'm sure it is quite simple and I've seen it somewhere before. I just can't recollect what it is.

[Martin_fr]

The only way of serious profiling I have used is valgrind/kcachegrind => Linux only.
And mind, the profiled app will run crawling slow... (but you can start it on the running app, e.g. on Lazarus just before you press F9)

But, also you would need to profile Lazarus and Fpc separately. And then loose info when one waits for the other.
Also, I doubt you will find any low hanging fruit, ...

The compiler is started by Lazarus itself. Lazarus is reading its StdOut via pipe. So compilation speed should be mostly down to FPC. If you have several packages, then there is a real tiny lag between one ending, and the next being started.

And then when the compiler is done, Lazarus goes directly to debugging. With gdb, it needs to run gdb (and as I said, that is slow / not gdb, but IDE<>gdb combo). With Fpdebug it is all build in.


Have you checked which of the 3 phases takes the time?

[dbannon]

27k lines of code and comments, hardware a bit like vfclists, Linux
Build - 4 seconds.
Cleanup & Build - 19 seconds

Davo

[KodeZwerg]

By reading your needs for an extreme fast compilation "server", you first need to invest, not into processor or storage, in RAM.
When that is done you need to setup a RAM-Drive, find out in your file explorer how big the installation for lazarus including fpc is and set the RAM-Drive to that ammount + 1GB extra.
When that is done download and run FpUp(deluxe) to install a fresh version on that RAM-Drive (this step is needed to setup FPC configuration correct to use RAM-Drive)
When that is done, install via OPM all needed packages.
When that is done, configure your fresh Lazarus IDE that all suits perfect for you.
When that is done, copy complete RAM-Drive to HDD (the fastest you have)
When that is done write a startup-script so when your OS is booted that the content from HDD is copied to the RAM-Drive.
Start Lazarus via the Desktop link that FpUp(deluxe) created and since the configuration will be the one that you made earlier, whatever you change now within Lazarus, after next boot that setting is lost.
Now you have the fastest version available and you do not need to buy a new hardware, beside stocking up RAM.
Remember: RAM-Drives are flushed when your Hardware crash or reboot etc... so make periodic backups of your RAM-Drive "projects"-folder to your HDD.

[Martin_fr]

With 16 GB of ram he might be able to do that.  Laz and Fpc should fit into less that 2GB, so if he makes it 4GB then thats fine. (And the fpc sources don't need to go there, just the ppu and o files)

Mind, I tried that long time ago => wasn't that amazingly better. (but I don't recall numbers. And there may be differences in drivers for the ram disk...

[dbannon]

When that is done you need to setup a RAM-Drive, find out in your file explorer how big the installation for lazarus including fpc is and set the RAM-Drive to that ammount + 1GB extra.

Given how well Unix uses its disk buffers, I doubt that would have much benefit to be honest. Its already Nvme2 and while compiling does trigger some "device contention", not much, its just about "RAM" anyway.  Unless, as TRon mentioned, the hardware setup is less than optimal. Thats unlikely with a box setup with 16G ram.

vfclists - can you give us your definition of "slow" - how long to Build so many lines of code ?

Davo