Programming language comparison by implementing the same transit data app

[avk]

It is a pity, of course, but from time to time there are cases when even the native compiler flatly refuses to compile LGenerics.
To be honest, I have never even tried to build FPC LLVM.

[cpicanco]

TLDR

Leledumbo

Just wondering what is the analysis/profiling method used in this performance optimization fun.

Does it take into account, for example, Emery Berger's notes (https://www.youtube.com/watch?v=r-TLSBdHe1A)?

***

This video https://www.youtube.com/watch?v=r-TLSBdHe1A has a talk by Emery Berger, it contains topics such as:

- performance driven development was boosted by 2000's, after clock speed reaching asymptotic levels despite hardware upgrades;

- performance optimization is very hard without proper analysis and profiling tools;

- layout biases measurements shows that speed can be affected by a lot of non-code related stuff, including changes in executable directory;

- link order (changes function adresses) and enviroment variable size (moves the program stack) has +-40% more speed impact than -O3;

- and so on

[abouchez]

@cpicanco
Yes, we know about this video. It is very funny, but with some quirks / oversimplifications.
This is why benchmarks, especially micro benchmarks, are not very reliable.
TLWR: Only real profiling makes sense to find the bottlenecks, not guessing.

In mORMot, we mostly optimized the core from actual performance data taken on production sites, doing real work.
The mormot.core.log unit has built-in features to measure time elapsed on real work, and make true profiling.
And our regression tests are not just unit tests, but they have a lot of real-world-similar scenarios, to have some good hint about potential performance regressions (or optimizations).

For FPC, the fact that its compiler and linker parts are simpler/less optimized than gcc/llvm counterparts makes it less likely to suffer from layout biases.
But, with the challenge involved in this forum thread, when you come from 26MB/s to 900MB/s, or when you see 10 times less memory usage running the benchmark on this thread, it is fair to say that the comparison means something.

[PascalDragon]

I just built FPC LLVM from source with LLVM 14. It doesn't compile the lgenerics library:

Code: Text  [Select][+][-]

1. % fpc -Clv14.0 -Fulgenerics app.pas
2. Free Pascal Compiler version 3.3.1 [2022/12/10] for x86_64
3. Copyright (c) 1993-2022 by Florian Klaempfl and others
4. Target OS: Linux for x86-64
5. Compiling app.pas
6. Compiling ./lgenerics/lgutils.pas
7. Compiling ./lgenerics/lgstrconst.pas
8. Writing Resource String Table file: lgstrconst.rsj
9. Assembling lgstrconst
10. lgutils.pas(3321,26) Warning: Function result variable of a managed type does not seem to be initialized
11. Assembling lgutils
12. Compiling ./lgenerics/lghashmap.pas
13. Compiling ./lgenerics/lghelpers.pas
14. Compiling ./lgenerics/lghash.pas
15. lghash.pas(442,35) Fatal: Internal error 200611011
16. Fatal: Compilation aborted
17. Error: /home/pierce/pkg/fpcllvm/lib/fpc/3.3.1/ppcx64 returned an error exitcode

 

This is the second program I'm building with FPC LLVM. The first one, hello world , builds and runs.

Best report this with a small, self contained example (and maybe also test whether it happens with non-LLVM FPC of the same commit as well).

[Leledumbo]

Doesn't it bother you that manual serialization in this case looks like an obvious cheating?

Not really sure why, after all the goal of this language comparison is to compare possible solutions in each language, so if the language can do it, it's not cheating. At least that's my interpretation.

[avk]

I added two versions of trascal to the example/json/transit-lang-cmp folder, the first(app2.pas) uses manual serialization and the second(app3.pas) works in much the same way as the Go version. I would like to note that the lgPdo unit still requires extensive testing.
You just need to update your LGenerics repository.