Survey: effect of the new heap.inc on the compiler.

[runewalsh]

If you use trunk FPC and have a large enough project that requires several seconds to compile, try the following:

— rebuild (Shift-F9 in Lazarus) your project with FPC @ 2a30ef7acac482f88402f7ebfbc582f6edfb2393 several times, note the minimum time,

— rebuild your project with FPC @ 7e0e284fce59b7e1a0f7638db205bcc9bbb576cf several times, note the minimum time,

— post both values.

My results seem to be:

[Martin_fr]

Not a compiler run. But part of some testcases I have. FPC/rtl and project compiled with -O4


From the FpDebug testcase (running both debugger code, and lots of string manipulation/comparison to check results):
Before: 16.140
After: 15.063

From the SynEdit testcase (2 tests)
Before 00:20.526 and 01:44.762
After  00:20.363 and 01:43.696


-- EDIT:
From IdeDebugger test (no significant difference)
Before 02:47.766
After 02:47.112

This test is doing a lot of mem alloc/dealloc. Mostly small objects, strings (xml) and a few arrays

[jamie]

I don't worry that much about compile time if the compiler is actually doing some work to make the end product execute better and not get carried away with large in the code.

 Better code analyzing normally requires a little more work to make the best choice of generated code.

 It's the end results that matters.

 Jamie

[AlexTP]

CudaText with 'JSON lite lexer', loading 170 Mb JSON file (dummy, 79K lines).

before: ~5.3s
now: ~4.2s

Good work!

[runewalsh]

I’m more interested in compilation time than everything else, because various programs can have various shares of the memory manager usage all the way down to 0%, while compiler is a known reference, and can still vary depending on the project.

My FPC compiles itself in 8.9 → 7.1 s.

[Martin_fr]

Well, maybe if I find more time. Compiler is much harder to compare, since it depends on disk i/o.

Even if multiple runs may filter out spikes, the results are also hard to interpret. Say something takes 100 seconds with the old, and  98 with the new mem manager to compile. What does it mean?

Well if I had zero wait time on disk i/o then that is a 2% gain.

But if I had a slower disk, and 90 seconds were idle waiting for disk io, then it went from 10 seconds of actual work, down to 8 seconds. That would be 20%.

On Linux "time" gives actual processing time (I think? So filtering out the wait?). On Windows I don't know if there is such a tool.

[runewalsh]

Windows (with enough “empty” RAM) more or less caches everything after the first run.

[Martin_fr]

Here is the output of building the IDE.

Each compiler had 4 runs, the first was discarded. (I have enough mem, to fit all files into a fraction of it / if Windows indeed does).

There is a one line change in unit graphics, as it forwards to the rtl, but that only works with the NEW compiler / the old doesn't have that code.

Also I forgot to limit the build to a single thread. So the FIRST TWO build OF EACH where running all core (packages building in parallel).
The last (right most) of each compiler was running single thread.

The very last line is the IDE code itself, which only starts when all packages are build. So that run single thread in all cases.

Code: Text  [Select][+][-]

1. New Mem                                                     Old mem
2.    197 lines compiled, 0.1 sec  , 0.1 sec  , 0.1 sec   |||     197 lines compiled, 0.1 sec  , 0.1 sec  , 0.1 sec
3.  87997 lines compiled, 2.6 sec  , 2.5 sec  , 2.2 sec   |||   87997 lines compiled, 2.7 sec  , 2.8 sec  , 2.4 sec
4.   1527 lines compiled, 0.2 sec  , 0.1 sec  , 0.5 sec   |||    1527 lines compiled, 0.1 sec  , 0.2 sec  , 0.6 sec
5.  14026 lines compiled, 1.0 sec  , 0.8 sec  , 8.3 sec   |||   14026 lines compiled, 0.9 sec  , 0.9 sec  ,10.3 sec
6.   1039 lines compiled, 0.2 sec  , 0.2 sec  , 1.4 sec   |||    1039 lines compiled, 0.2 sec  , 0.2 sec  , 1.5 sec
7.  24115 lines compiled, 1.1 sec  , 1.1 sec  , 1.5 sec   |||   24115 lines compiled, 1.2 sec  , 1.0 sec  , 1.5 sec
8. 139287 lines compiled, 5.4 sec  , 5.7 sec  , 3.8 sec   |||  139287 lines compiled, 6.1 sec  , 6.0 sec  , 4.5 sec
9.   9878 lines compiled, 1.0 sec  , 1.0 sec  , 5.8 sec   |||    9878 lines compiled, 1.1 sec  , 0.9 sec  , 7.0 sec
10.   7357 lines compiled, 0.6 sec  , 0.7 sec  , 0.9 sec   |||    7357 lines compiled, 0.6 sec  , 0.5 sec  , 1.0 sec
11.  35784 lines compiled, 2.1 sec  , 2.1 sec  , 0.5 sec   |||   35784 lines compiled, 2.3 sec  , 2.2 sec  , 0.5 sec
12. 225571 lines compiled, 9.3 sec  , 9.4 sec  , 0.1 sec   |||  225571 lines compiled,11.2 sec  ,11.3 sec  , 0.1 sec
13.  16475 lines compiled, 0.9 sec  , 0.9 sec  , 0.3 sec   |||   16475 lines compiled, 0.9 sec  , 0.9 sec  , 0.3 sec
14.  30227 lines compiled, 1.6 sec  , 1.6 sec  , 3.2 sec   |||   30227 lines compiled, 1.8 sec  , 1.8 sec  , 3.8 sec
15. 211252 lines compiled, 1.7 sec  , 1.7 sec  , 4.1 sec   |||  211252 lines compiled, 1.8 sec  , 1.8 sec  , 4.6 sec
16.  71649 lines compiled, 7.6 sec  , 7.6 sec  , 2.4 sec   |||   71649 lines compiled, 8.7 sec  , 8.5 sec  , 2.7 sec
17.   4766 lines compiled, 0.5 sec  , 0.6 sec  , 0.3 sec   |||    4766 lines compiled, 0.6 sec  , 0.5 sec  , 0.3 sec
18.   1899 lines compiled, 0.4 sec  , 0.4 sec  , 1.2 sec   |||    1899 lines compiled, 0.4 sec  , 0.5 sec  , 1.2 sec
19.  12093 lines compiled, 2.0 sec  , 2.1 sec  , 0.9 sec   |||   12093 lines compiled, 2.2 sec  , 2.3 sec  , 0.9 sec
20. 139158 lines compiled, 5.2 sec  , 5.0 sec  , 0.4 sec   |||  139158 lines compiled, 5.6 sec  , 5.6 sec  , 0.4 sec
21.    860 lines compiled, 0.6 sec  , 0.6 sec  , 0.9 sec   |||     860 lines compiled, 0.6 sec  , 0.6 sec  , 0.9 sec
22.   1804 lines compiled, 0.5 sec  , 0.4 sec  , 0.4 sec   |||    1804 lines compiled, 0.5 sec  , 0.5 sec  , 0.4 sec
23.  19474 lines compiled, 1.0 sec  , 1.2 sec  , 1.0 sec   |||   19474 lines compiled, 1.2 sec  , 1.1 sec  , 1.1 sec
24.  11651 lines compiled, 1.6 sec  , 1.6 sec  , 1.0 sec   |||   11651 lines compiled, 1.7 sec  , 1.8 sec  , 1.0 sec
25.  10424 lines compiled, 0.9 sec  , 1.0 sec  , 1.4 sec   |||   10424 lines compiled, 1.1 sec  , 1.0 sec  , 1.4 sec
26.  90677 lines compiled, 3.5 sec  , 3.5 sec  , 6.6 sec   |||   90677 lines compiled, 3.8 sec  , 3.9 sec  , 7.3 sec
27.    760 lines compiled, 0.6 sec  , 0.6 sec  , 0.2 sec   |||     760 lines compiled, 0.6 sec  , 0.5 sec  , 0.2 sec
28.  14746 lines compiled, 1.5 sec  , 1.4 sec  , 0.3 sec   |||   14746 lines compiled, 1.7 sec  , 1.6 sec  , 0.3 sec
29. 122282 lines compiled,11.5 sec  ,11.0 sec  , 1.1 sec   |||  122282 lines compiled,13.6 sec  ,13.7 sec  , 1.1 sec
30.   2543 lines compiled, 0.7 sec  , 0.7 sec  , 1.3 sec   |||    2543 lines compiled, 0.8 sec  , 0.8 sec  , 1.3 sec
31.  33250 lines compiled, 2.9 sec  , 3.0 sec  , 2.8 sec   |||   33250 lines compiled, 3.4 sec  , 3.3 sec  , 3.0 sec
32.  47382 lines compiled, 5.3 sec  , 5.2 sec  , 1.2 sec   |||   47382 lines compiled, 6.2 sec  , 6.0 sec  , 1.3 sec
33.   3324 lines compiled, 1.2 sec  , 1.2 sec  , 1.4 sec   |||    3324 lines compiled, 1.4 sec  , 1.4 sec  , 1.5 sec
34.  39416 lines compiled, 2.8 sec  , 2.6 sec  , 0.5 sec   |||   39416 lines compiled, 2.8 sec  , 2.9 sec  , 0.5 sec
35.   2448 lines compiled, 1.2 sec  , 0.8 sec  , 0.7 sec   |||    2448 lines compiled, 0.9 sec  , 0.9 sec  , 0.7 sec
36.   1333 lines compiled, 1.2 sec  , 0.9 sec  , 0.6 sec   |||    1333 lines compiled, 0.9 sec  , 0.9 sec  , 0.6 sec
37.  39367 lines compiled, 2.6 sec  , 2.3 sec  , 1.3 sec   |||   39367 lines compiled, 2.3 sec  , 2.5 sec  , 1.5 sec
38.  44796 lines compiled, 7.7 sec  , 6.7 sec  , 0.5 sec   |||   44796 lines compiled, 7.7 sec  , 7.7 sec  , 0.5 sec
39.   7499 lines compiled, 3.8 sec  , 3.0 sec  , 1.0 sec   |||    7499 lines compiled, 3.5 sec  , 3.8 sec  , 1.1 sec
40.    154 lines compiled, 0.9 sec  , 0.8 sec  , 1.4 sec   |||     154 lines compiled, 0.8 sec  , 0.8 sec  , 1.5 sec
41.   4606 lines compiled, 1.5 sec  , 1.2 sec  , 0.4 sec   |||    4606 lines compiled, 1.4 sec  , 1.4 sec  , 0.4 sec
42.   3567 lines compiled, 1.7 sec  , 1.3 sec  , 1.0 sec   |||    3567 lines compiled, 1.4 sec  , 1.4 sec  , 1.0 sec
43.   1789 lines compiled, 1.5 sec  , 0.9 sec  , 0.3 sec   |||    1789 lines compiled, 1.0 sec  , 1.0 sec  , 0.3 sec
44.   1439 lines compiled, 2.2 sec  , 1.9 sec  , 1.3 sec   |||    1439 lines compiled, 2.2 sec  , 2.4 sec  , 1.4 sec
45.   1198 lines compiled, 1.1 sec  , 0.8 sec  , 0.1 sec   |||    1198 lines compiled, 0.8 sec  , 0.9 sec  , 0.2 sec
46.   1196 lines compiled, 1.5 sec  , 1.1 sec  , 0.4 sec   |||    1196 lines compiled, 1.3 sec  , 1.3 sec  , 0.5 sec
47.    590 lines compiled, 1.4 sec  , 0.8 sec  , 0.6 sec   |||     590 lines compiled, 0.9 sec  , 2.9 sec  , 0.6 sec
48.    406 lines compiled, 0.9 sec  , 2.2 sec  , 0.6 sec   |||    6838 lines compiled, 2.5 sec  , 1.0 sec  , 0.6 sec
49.    353 lines compiled, 0.7 sec  , 0.8 sec  , 0.5 sec   |||     406 lines compiled, 0.7 sec  , 0.9 sec  , 0.5 sec
50.    152 lines compiled, 1.2 sec  , 0.7 sec  , 1.1 sec   |||     353 lines compiled, 0.7 sec  , 0.9 sec  , 1.2 sec
51.   6838 lines compiled, 2.5 sec  , 0.9 sec  , 1.1 sec   |||     152 lines compiled, 1.0 sec  , 1.1 sec  , 1.1 sec
52.  13290 lines compiled, 1.9 sec  , 2.1 sec  , 1.2 sec   |||   13290 lines compiled, 2.2 sec  , 2.2 sec  , 1.2 sec
53.  22312 lines compiled, 2.5 sec  , 2.7 sec  , 0.3 sec   |||   22312 lines compiled, 2.8 sec  , 2.8 sec  , 0.3 sec
54.   9049 lines compiled, 3.0 sec  , 3.0 sec  , 0.4 sec   |||    9049 lines compiled, 3.4 sec  , 3.3 sec  , 0.4 sec
55.   5096 lines compiled, 0.9 sec  , 0.8 sec  , 0.9 sec   |||    5096 lines compiled, 0.8 sec  , 0.9 sec  , 1.0 sec
56.   2655 lines compiled, 2.4 sec  , 3.7 sec  , 1.0 sec   |||   16729 lines compiled, 4.1 sec  , 4.0 sec  , 1.1 sec
57.   2006 lines compiled, 2.0 sec  , 2.3 sec  , 0.3 sec   |||    2655 lines compiled, 2.7 sec  , 2.5 sec  , 0.3 sec
58.  16729 lines compiled, 3.9 sec  , 2.2 sec  , 1.1 sec   |||    2006 lines compiled, 2.5 sec  , 2.1 sec  , 1.1 sec
59.  12295 lines compiled, 2.5 sec  , 2.9 sec  , 0.4 sec   |||   12295 lines compiled, 2.5 sec  , 2.3 sec  , 0.4 sec
60.   8694 lines compiled, 2.8 sec  , 3.1 sec  , 0.5 sec   |||    8694 lines compiled, 2.9 sec  , 2.9 sec  , 0.5 sec
61.   2247 lines compiled, 3.5 sec  , 2.7 sec  , 1.0 sec   |||    2247 lines compiled, 4.0 sec  , 3.3 sec  , 1.1 sec
62.   2902 lines compiled, 2.9 sec  , 2.2 sec  , 0.2 sec   |||    2902 lines compiled, 3.1 sec  , 2.9 sec  , 0.2 sec
63.   2070 lines compiled, 3.2 sec  , 2.8 sec  , 1.0 sec   |||    2070 lines compiled, 3.0 sec  , 3.2 sec  , 1.1 sec
64.   1333 lines compiled, 2.3 sec  , 2.6 sec  , 0.5 sec   |||    1333 lines compiled, 2.8 sec  , 2.8 sec  , 0.2 sec
65.    982 lines compiled, 3.0 sec  , 2.2 sec  , 0.4 sec   |||     982 lines compiled, 2.8 sec  , 2.7 sec  , 0.4 sec
66.    787 lines compiled, 3.0 sec  , 2.1 sec  , 0.3 sec   |||     787 lines compiled, 2.7 sec  , 2.6 sec  , 0.3 sec
67.    285 lines compiled, 2.5 sec  , 2.2 sec  , 0.3 sec   |||     285 lines compiled, 2.6 sec  , 2.5 sec  , 0.3 sec
68.  48354 lines compiled, 7.8 sec  , 0.4 sec  , 1.0 sec   |||   48354 lines compiled, 9.1 sec  , 0.5 sec  , 1.1 sec
69.    812 lines compiled, 0.4 sec  , 0.7 sec  , 0.4 sec   |||     812 lines compiled, 0.5 sec  , 0.9 sec  , 0.4 sec
70.    638 lines compiled, 0.5 sec  , 0.6 sec  , 0.6 sec   |||     638 lines compiled, 0.8 sec  , 0.6 sec  , 0.6 sec
71.     71 lines compiled, 0.5 sec  , 7.6 sec  , 0.2 sec   |||      71 lines compiled, 0.6 sec  , 8.4 sec  , 0.2 sec
72. 251269 lines compiled,26.5 sec  ,26.5 sec  ,26.2 sec   |||  251269 lines compiled,35.3 sec  ,35.8 sec  ,35.2 sec
73. , 22175568 bytes code  , 22175568          , 22175568       , 22170064 bytes code       , 22170064     , 22170064
74. , 1188164 bytes data   , 1188164           , 1188164        , 1188100 bytes data        , 1188100      , 1188100  
75.    

[Martin_fr]

Just noticed, that also means that the order is different in each run.

It should be the same order for the right most column of each... (Unless the IDE build system did something unexpected). The line counts are from the first (left most) run => so you can sort that for the left most run if you want....

Well, the upper half seems to have kept same order. Only the lower entries eventually got their own order (depending on how quick other packages finished)

Sorry, about that. But not re-running now. Takes quite some time.

--- EDIT

Below are all outputs (one missing) before putting them together.

So if you want to look closer you can sort each by the amount of lines, and then you have all of them in the same order.

Code: Text  [Select][+][-]

1. New Mem
2.    197 lines compiled, 0.1 sec
3.  87997 lines compiled, 2.6 sec
4.   1527 lines compiled, 0.2 sec
5.  14026 lines compiled, 1.0 sec
6.   1039 lines compiled, 0.2 sec
7.  24115 lines compiled, 1.1 sec
8. 139287 lines compiled, 5.4 sec
9.   9878 lines compiled, 1.0 sec
10.   7357 lines compiled, 0.6 sec
11.  35784 lines compiled, 2.1 sec
12. 225571 lines compiled, 9.3 sec
13.  16475 lines compiled, 0.9 sec
14.  30227 lines compiled, 1.6 sec
15. 211252 lines compiled, 1.7 sec
16.  71649 lines compiled, 7.6 sec
17.   4766 lines compiled, 0.5 sec
18.   1899 lines compiled, 0.4 sec
19.  12093 lines compiled, 2.0 sec
20. 139158 lines compiled, 5.2 sec
21.    860 lines compiled, 0.6 sec
22.   1804 lines compiled, 0.5 sec
23.  19474 lines compiled, 1.0 sec
24.  11651 lines compiled, 1.6 sec
25.  10424 lines compiled, 0.9 sec
26.  90677 lines compiled, 3.5 sec
27.    760 lines compiled, 0.6 sec
28.  14746 lines compiled, 1.5 sec
29. 122282 lines compiled,11.5 sec
30.   2543 lines compiled, 0.7 sec
31.  33250 lines compiled, 2.9 sec
32.  47382 lines compiled, 5.3 sec
33.   3324 lines compiled, 1.2 sec
34.  39416 lines compiled, 2.8 sec
35.   2448 lines compiled, 1.2 sec
36.   1333 lines compiled, 1.2 sec
37.  39367 lines compiled, 2.6 sec
38.  44796 lines compiled, 7.7 sec
39.   7499 lines compiled, 3.8 sec
40.    154 lines compiled, 0.9 sec
41.   4606 lines compiled, 1.5 sec
42.   3567 lines compiled, 1.7 sec
43.   1789 lines compiled, 1.5 sec
44.   1439 lines compiled, 2.2 sec
45.   1198 lines compiled, 1.1 sec
46.   1196 lines compiled, 1.5 sec
47.    590 lines compiled, 1.4 sec
48.    406 lines compiled, 0.9 sec
49.    353 lines compiled, 0.7 sec
50.    152 lines compiled, 1.2 sec
51.   6838 lines compiled, 2.5 sec
52.  13290 lines compiled, 1.9 sec
53.  22312 lines compiled, 2.5 sec
54.   9049 lines compiled, 3.0 sec
55.   5096 lines compiled, 0.9 sec
56.   2655 lines compiled, 2.4 sec
57.   2006 lines compiled, 2.0 sec
58.  16729 lines compiled, 3.9 sec
59.  12295 lines compiled, 2.5 sec
60.   8694 lines compiled, 2.8 sec
61.   2247 lines compiled, 3.5 sec
62.   2902 lines compiled, 2.9 sec
63.   2070 lines compiled, 3.2 sec
64.   1333 lines compiled, 2.3 sec
65.    982 lines compiled, 3.0 sec
66.    787 lines compiled, 3.0 sec
67.    285 lines compiled, 2.5 sec
68.  48354 lines compiled, 7.8 sec
69.    812 lines compiled, 0.4 sec
70.    638 lines compiled, 0.5 sec
71.     71 lines compiled, 0.5 sec
72. 251269 lines compiled,26.5 sec
73. , 22175568 bytes code, 1188164 bytes data
74.  
75.    197 lines compiled, 0.1 sec
76.  87997 lines compiled, 2.5 sec
77.   1527 lines compiled, 0.1 sec
78.  14026 lines compiled, 0.8 sec
79.   1039 lines compiled, 0.2 sec
80.  24115 lines compiled, 1.1 sec
81. 139287 lines compiled, 5.7 sec
82.   9878 lines compiled, 1.0 sec
83.   7357 lines compiled, 0.7 sec
84.  35784 lines compiled, 2.1 sec
85. 225571 lines compiled, 9.4 sec
86.  16475 lines compiled, 0.9 sec
87.  30227 lines compiled, 1.6 sec
88. 211252 lines compiled, 1.7 sec
89.  71649 lines compiled, 7.6 sec
90.   4766 lines compiled, 0.6 sec
91.   1899 lines compiled, 0.4 sec
92.  12093 lines compiled, 2.1 sec
93. 139158 lines compiled, 5.0 sec
94.    860 lines compiled, 0.6 sec
95.   1804 lines compiled, 0.4 sec
96.  19474 lines compiled, 1.2 sec
97.  11651 lines compiled, 1.6 sec
98.  10424 lines compiled, 1.0 sec
99.  90677 lines compiled, 3.5 sec
100.    760 lines compiled, 0.6 sec
101.  14746 lines compiled, 1.4 sec
102. 122282 lines compiled,11.0 sec
103.   2543 lines compiled, 0.7 sec
104.  33250 lines compiled, 3.0 sec
105.  47382 lines compiled, 5.2 sec
106.   3324 lines compiled, 1.2 sec
107.  39416 lines compiled, 2.6 sec
108.   2448 lines compiled, 0.8 sec
109.   1333 lines compiled, 0.9 sec
110.  39367 lines compiled, 2.3 sec
111.  44796 lines compiled, 6.7 sec
112.   7499 lines compiled, 3.0 sec
113.    154 lines compiled, 0.8 sec
114.   4606 lines compiled, 1.2 sec
115.   3567 lines compiled, 1.3 sec
116.   1789 lines compiled, 0.9 sec
117.   1439 lines compiled, 1.9 sec
118.   1198 lines compiled, 0.8 sec
119.   1196 lines compiled, 1.1 sec
120.    590 lines compiled, 0.8 sec
121.   6838 lines compiled, 2.2 sec
122.    406 lines compiled, 0.8 sec
123.    353 lines compiled, 0.7 sec
124.    152 lines compiled, 0.9 sec
125.  13290 lines compiled, 2.1 sec
126.  22312 lines compiled, 2.7 sec
127.   9049 lines compiled, 3.0 sec
128.   5096 lines compiled, 0.8 sec
129.  16729 lines compiled, 3.7 sec
130.   2655 lines compiled, 2.3 sec
131.   2006 lines compiled, 2.2 sec
132.   8694 lines compiled, 2.9 sec
133.   2247 lines compiled, 3.1 sec
134.   2902 lines compiled, 2.7 sec
135.  12295 lines compiled, 2.2 sec
136.   2070 lines compiled, 2.8 sec
137.    982 lines compiled, 2.6 sec
138.    787 lines compiled, 2.2 sec
139.   1333 lines compiled, 2.1 sec
140.    285 lines compiled, 2.2 sec
141.    812 lines compiled, 0.4 sec
142.    638 lines compiled, 0.7 sec
143.     71 lines compiled, 0.6 sec
144.  48354 lines compiled, 7.6 sec
145. 251269 lines compiled,26.5 sec
146. , 22175568 bytes code, 1188164 bytes data
147.  
148.  
149.  
150. single thread
151.    197 lines compiled, 0.1 sec
152.  87997 lines compiled, 2.2 sec
153.  24115 lines compiled, 0.5 sec
154. 225571 lines compiled, 8.3 sec
155.  30227 lines compiled, 1.4 sec
156. 211252 lines compiled, 1.5 sec
157. 139287 lines compiled, 3.8 sec
158. 122282 lines compiled, 5.8 sec
159.  12093 lines compiled, 0.9 sec
160.   9878 lines compiled, 0.5 sec
161.   1527 lines compiled, 0.1 sec
162.   7357 lines compiled, 0.3 sec
163.  47382 lines compiled, 3.2 sec
164.  71649 lines compiled, 4.1 sec
165. 139158 lines compiled, 2.4 sec
166.    860 lines compiled, 0.3 sec
167.  39416 lines compiled, 1.2 sec
168.  16475 lines compiled, 0.9 sec
169.   2448 lines compiled, 0.4 sec
170.  13290 lines compiled, 0.9 sec
171.   1333 lines compiled, 0.4 sec
172.  22312 lines compiled, 1.0 sec
173.  12295 lines compiled, 1.0 sec
174.   9049 lines compiled, 1.4 sec
175.  48354 lines compiled, 6.6 sec
176.   1804 lines compiled, 0.2 sec
177.   2543 lines compiled, 0.3 sec
178.  39367 lines compiled, 1.1 sec
179.  35784 lines compiled, 1.3 sec
180.  44796 lines compiled, 2.8 sec
181.  33250 lines compiled, 1.2 sec
182.  16729 lines compiled, 1.4 sec
183.  19474 lines compiled, 0.5 sec
184.  11651 lines compiled, 0.7 sec
185.  14026 lines compiled, 0.6 sec
186.   7499 lines compiled, 1.3 sec
187.  10424 lines compiled, 0.5 sec
188.   8694 lines compiled, 1.0 sec
189.  90677 lines compiled, 1.4 sec
190.    154 lines compiled, 0.4 sec
191.   6838 lines compiled, 1.0 sec
192.   5096 lines compiled, 0.3 sec
193.   2247 lines compiled, 1.3 sec
194.   1039 lines compiled, 0.1 sec
195.   4766 lines compiled, 0.4 sec
196.   4606 lines compiled, 0.6 sec
197.   3567 lines compiled, 0.6 sec
198.   3324 lines compiled, 0.5 sec
199.   2902 lines compiled, 1.1 sec
200.   2655 lines compiled, 1.1 sec
201.   2070 lines compiled, 1.2 sec
202.    760 lines compiled, 0.3 sec
203.   1789 lines compiled, 0.4 sec
204.   2006 lines compiled, 0.9 sec
205.   1439 lines compiled, 1.0 sec
206.   1899 lines compiled, 0.3 sec
207.   1333 lines compiled, 1.1 sec
208.   1198 lines compiled, 0.4 sec
209.   1196 lines compiled, 0.5 sec
210.    982 lines compiled, 1.0 sec
211.    812 lines compiled, 0.2 sec
212.    787 lines compiled, 1.0 sec
213.    638 lines compiled, 0.5 sec
214.    590 lines compiled, 0.4 sec
215.    406 lines compiled, 0.3 sec
216.    353 lines compiled, 0.3 sec
217.    285 lines compiled, 1.0 sec
218.    152 lines compiled, 0.4 sec
219.  14746 lines compiled, 0.6 sec
220.     71 lines compiled, 0.2 sec
221. 251269 lines compiled,26.2 sec
222. , 22175568 bytes code, 1188164 bytes data
223.  
224. ======================================
225. Old mem
226.    197 lines compiled, 0.1 sec  d, 0.1 sec
227.  87997 lines compiled, 2.7 sec  d, 2.8 sec
228.   1527 lines compiled, 0.1 sec  d, 0.2 sec
229.  14026 lines compiled, 0.9 sec  d, 0.9 sec
230.   1039 lines compiled, 0.2 sec  d, 0.2 sec
231.  24115 lines compiled, 1.2 sec  d, 1.0 sec
232. 139287 lines compiled, 6.1 sec  d, 6.0 sec
233.   9878 lines compiled, 1.1 sec  d, 0.9 sec
234.   7357 lines compiled, 0.6 sec  d, 0.5 sec
235.  35784 lines compiled, 2.3 sec  d, 2.2 sec
236. 225571 lines compiled,11.2 sec  d,11.3 sec
237.  16475 lines compiled, 0.9 sec  d, 0.9 sec
238.  30227 lines compiled, 1.8 sec  d, 1.8 sec
239. 211252 lines compiled, 1.8 sec  d, 1.8 sec
240.  71649 lines compiled, 8.7 sec  d, 8.5 sec
241.   4766 lines compiled, 0.6 sec  d, 0.5 sec
242.   1899 lines compiled, 0.4 sec  d, 0.5 sec
243.  12093 lines compiled, 2.2 sec  d, 2.3 sec
244. 139158 lines compiled, 5.6 sec  d, 5.6 sec
245.    860 lines compiled, 0.6 sec  d, 0.6 sec
246.   1804 lines compiled, 0.5 sec  d, 0.5 sec
247.  19474 lines compiled, 1.2 sec  d, 1.1 sec
248.  11651 lines compiled, 1.7 sec  d, 1.8 sec
249.  10424 lines compiled, 1.1 sec  d, 1.0 sec
250.  90677 lines compiled, 3.8 sec  d, 3.9 sec
251.    760 lines compiled, 0.6 sec  d, 0.5 sec
252.  14746 lines compiled, 1.7 sec  d, 1.6 sec
253. 122282 lines compiled,13.6 sec  d,13.7 sec
254.   2543 lines compiled, 0.8 sec  d, 0.8 sec
255.  33250 lines compiled, 3.4 sec  d, 3.3 sec
256.  47382 lines compiled, 6.2 sec  d, 6.0 sec
257.   3324 lines compiled, 1.4 sec  d, 1.4 sec
258.  39416 lines compiled, 2.8 sec  d, 2.9 sec
259.   2448 lines compiled, 0.9 sec  d, 0.9 sec
260.   1333 lines compiled, 0.9 sec  d, 0.9 sec
261.  39367 lines compiled, 2.3 sec  d, 2.5 sec
262.  44796 lines compiled, 7.7 sec  d, 7.7 sec
263.   7499 lines compiled, 3.5 sec  d, 3.8 sec
264.    154 lines compiled, 0.8 sec  d, 0.8 sec
265.   4606 lines compiled, 1.4 sec  d, 1.4 sec
266.   3567 lines compiled, 1.4 sec  d, 1.4 sec
267.   1789 lines compiled, 1.0 sec  d, 1.0 sec
268.   1439 lines compiled, 2.2 sec  d, 2.4 sec
269.   1198 lines compiled, 0.8 sec  d, 0.9 sec
270.   1196 lines compiled, 1.3 sec  d, 1.3 sec
271.    590 lines compiled, 0.9 sec  d, 2.9 sec
272.   6838 lines compiled, 2.5 sec  d, 1.0 sec
273.    406 lines compiled, 0.7 sec  d, 0.9 sec
274.    353 lines compiled, 0.7 sec  d, 0.9 sec
275.    152 lines compiled, 1.0 sec  d, 1.1 sec
276.  13290 lines compiled, 2.2 sec  d, 2.2 sec
277.  22312 lines compiled, 2.8 sec  d, 2.8 sec
278.   9049 lines compiled, 3.4 sec  d, 3.3 sec
279.   5096 lines compiled, 0.8 sec  d, 0.9 sec
280.  16729 lines compiled, 4.1 sec  d, 4.0 sec
281.   2655 lines compiled, 2.7 sec  d, 2.5 sec
282.   2006 lines compiled, 2.5 sec  d, 2.1 sec
283.  12295 lines compiled, 2.5 sec  d, 2.3 sec
284.   8694 lines compiled, 2.9 sec  d, 2.9 sec
285.   2247 lines compiled, 4.0 sec  d, 3.3 sec
286.   2902 lines compiled, 3.1 sec  d, 2.9 sec
287.   2070 lines compiled, 3.0 sec  d, 3.2 sec
288.   1333 lines compiled, 2.8 sec  d, 2.8 sec
289.    982 lines compiled, 2.8 sec  d, 2.7 sec
290.    787 lines compiled, 2.7 sec  d, 2.6 sec
291.    285 lines compiled, 2.6 sec  d, 2.5 sec
292.  48354 lines compiled, 9.1 sec  d, 0.5 sec
293.    812 lines compiled, 0.5 sec  d, 0.9 sec
294.    638 lines compiled, 0.8 sec  d, 0.6 sec
295.     71 lines compiled, 0.6 sec  d, 8.4 sec
296. 251269 lines compiled,35.3 sec  d,35.8 sec
297. , 22170064 bytes code           , 22170064 bytes code
298. , 1188100 bytes data            , 1188100 bytes data          
299.  
300.  
301.  
302.  
303. single thread
304.  
305.  
306.    197 lines compiled, 0.1 sec
307.  87997 lines compiled, 2.4 sec
308.  24115 lines compiled, 0.6 sec
309. 225571 lines compiled,10.3 sec
310.  30227 lines compiled, 1.5 sec
311. 211252 lines compiled, 1.5 sec
312. 139287 lines compiled, 4.5 sec
313. 122282 lines compiled, 7.0 sec
314.  12093 lines compiled, 1.0 sec
315.   9878 lines compiled, 0.5 sec
316.   1527 lines compiled, 0.1 sec
317.   7357 lines compiled, 0.3 sec
318.  47382 lines compiled, 3.8 sec
319.  71649 lines compiled, 4.6 sec
320. 139158 lines compiled, 2.7 sec
321.    860 lines compiled, 0.3 sec
322.  39416 lines compiled, 1.2 sec
323.  16475 lines compiled, 0.9 sec
324.   2448 lines compiled, 0.4 sec
325.  13290 lines compiled, 0.9 sec
326.   1333 lines compiled, 0.4 sec
327.  22312 lines compiled, 1.1 sec
328.  12295 lines compiled, 1.0 sec
329.   9049 lines compiled, 1.4 sec
330.  48354 lines compiled, 7.3 sec
331.   1804 lines compiled, 0.2 sec
332.   2543 lines compiled, 0.3 sec
333.  39367 lines compiled, 1.1 sec
334.  35784 lines compiled, 1.3 sec
335.  44796 lines compiled, 3.0 sec
336.  33250 lines compiled, 1.3 sec
337.  16729 lines compiled, 1.5 sec
338.  19474 lines compiled, 0.5 sec
339.  11651 lines compiled, 0.7 sec
340.  14026 lines compiled, 0.6 sec
341.   7499 lines compiled, 1.5 sec
342.  10424 lines compiled, 0.5 sec
343.   8694 lines compiled, 1.1 sec
344.  90677 lines compiled, 1.5 sec
345.    154 lines compiled, 0.4 sec
346.   6838 lines compiled, 1.0 sec
347.   5096 lines compiled, 0.3 sec
348.   2247 lines compiled, 1.4 sec
349.   1039 lines compiled, 0.2 sec
350.   4766 lines compiled, 0.5 sec
351.   4606 lines compiled, 0.6 sec
352.   3567 lines compiled, 0.6 sec
353.   3324 lines compiled, 0.5 sec
354.   2902 lines compiled, 1.2 sec
355.   2655 lines compiled, 1.1 sec
356.   2070 lines compiled, 1.2 sec
357.    760 lines compiled, 0.3 sec
358.   1789 lines compiled, 0.4 sec
359.   2006 lines compiled, 1.0 sec
360.   1439 lines compiled, 1.1 sec
361.   1899 lines compiled, 0.3 sec
362.   1333 lines compiled, 1.1 sec
363.   1198 lines compiled, 0.4 sec
364.   1196 lines compiled, 0.5 sec
365.    982 lines compiled, 1.1 sec
366.    812 lines compiled, 0.2 sec
367.    787 lines compiled, 1.1 sec
368.    638 lines compiled, 0.2 sec
369.    590 lines compiled, 0.4 sec
370.    406 lines compiled, 0.3 sec
371.    353 lines compiled, 0.3 sec
372.    285 lines compiled, 1.1 sec
373.    152 lines compiled, 0.4 sec
374.  14746 lines compiled, 0.6 sec
375.     71 lines compiled, 0.2 sec
376. 251269 lines compiled,35.2 sec
377. , 22170064 bytes code
378. , 1188100 bytes data                  

[Martin_fr]

because various programs can have various shares of the memory manager usage all the way down to 0%, while compiler is a known reference, and can still vary depending on the project

Btw, the "IdeDebugger test" should be doing plenty of mem alloc/dealloc.

It is all about testing to organize data in different structures (building lists of objects, and nested structures, but also a lot of text creating xml (both as TXmlDoc and as actual text in strings). It does build and release many hundred thousand of such structures.

Yet, it did not benefit.

The test is in the lazarus ide/package/idedebugger folder.

[PascalDragon]

I don't worry that much about compile time if the compiler is actually doing some work to make the end product execute better and not get carried away with large in the code.

This is not about changes in the compiler, but changes to the general heap that are used by both programs compiled by FPC and the compiler itself, thus using the compiler as a testcase to judge changes done to the heap is valid as it's a relatively memory heavy program.

[runewalsh]

ide/package/idedebugger indeed doesn’t benefit, but it’s okay, not all cases are made equal, the only thing they shouldn’t do is become (considerably) slower. In fact, it becomes slightly slower for me (by 2 s maybe, out of 190; I disabled heaptrc ofc) but again, −1% is okay (don’t even want to investigate ... ... ... micro-optimizations are possible but I will have a hard time proving them with this 1%).

[Martin_fr]

ide/package/idedebugger indeed doesn’t benefit, but it’s okay, not all cases are made equal, the only thing they shouldn’t do is become (considerably) slower. In fact, it becomes slightly slower for me (by 2 s maybe, out of 190; I disabled heaptrc ofc) but again, −1% is okay (don’t even want to investigate ... ... ... micro-optimizations are possible but I will have a hard time proving them with this 1%).

1% is no problem. Especially when it seems to even be machine dependent. That could easily be cache hits/misses if the mem layout changed. And, then the real application (not the testcase) would have its own mem layout anyway.

But this is why I added other tests.

Ideally we should see either
- no change (including plus or mines 2 or 3%)
- Improvements.

If there are reproducible cases that get a lot slower, then that would be worth noticing.

[marcov]

(Note that for building one should note if one uses Windows or not, as the windows internal linker probably also the memory manager)

[zamtmn]

I tried ZCAD build. (I had to rebuild lazarus too). It looks like the compiler is not a big consumer of dynamic memory

FPC @ 2a30ef7acac482f88402f7ebfbc582f6edfb2393

(9015) Linking Y:\zcad\cad\bin\x86_64-win64\zcad.exe
(1008) 159935 lines compiled, 49.9 sec, 10699760 bytes code, 588548 bytes data

FPC @ 7e0e284fce59b7e1a0f7638db205bcc9bbb576cf

(9015) Linking Y:\zcad\cad\bin\x86_64-win64\zcad.exe
(1008) 159935 lines compiled, 49.9 sec, 10705360 bytes code, 588580 bytes data

As for the work in ZCAD, I can say that there used to be a memory fragmentation effect. Loading drawings after selectively closing the previous ones took longer and longer each time. This is no longer the case. The drawing always loads in the same amount of time.