Lazarus

Using the Lazarus IDE => General => Topic started by: lainz on April 06, 2019, 11:20:43 pm

Title: Lazarus IDE with Cmem
Post by: lainz on April 06, 2019, 11:20:43 pm
Hi, I just rebuilt Lazarus IDE adding cmem unit right here in the Lazarus source:

Code: Pascal  [Select][+][-]
  1.   {$IFDEF EnableRedirectStdErr}
  2.   redirect_stderr,
  3.   {$ENDIF}
  4.   {$IF defined(HASAMIGA) and not defined(DisableMultiThreading)}
  5.   athreads,
  6.   {$ENDIF}
  7.   {$IF defined(UNIX) and not defined(DisableMultiThreading)}
  8.   cthreads,
  9.   {$ENDIF}
  10.   {$IFDEF IDE_MEM_CHECK}
  11.   MemCheck,
  12.   {$ENDIF}
  13.   {$IF defined(Unix)}
  14.   clocale,
  15.   {$IFEND}
  16.   cmem, // <---------- here
  17.  

I've tested this on Windows and works fine, a normal window was taking 58 mb of ram, now is using 32 mb of ram.

This of course has the side effect that I can not debug the IDE for memory leaks, but I don't do that so there's no problem.

It should work faster now? At least it uses less memory.
Title: Re: Lazarus IDE with Cmem
Post by: marcov on April 06, 2019, 11:37:59 pm
Cmem is not about faster. It is for when you communicate with libraries of "a" C compilre (usually the "default" for the platform), to use that C compiler's memory manager to harmonize memory use.

Whatever is under "cmem" depends on which C compiler (and library) you link to, and thus on what platform.
Title: Re: Lazarus IDE with Cmem
Post by: lainz on April 06, 2019, 11:41:08 pm
Alright  :)
Title: Re: Lazarus IDE with Cmem
Post by: marcov on April 06, 2019, 11:42:11 pm
I wouldn't use it on Windows. If you use it, keep it *nix only.
Title: Re: Lazarus IDE with Cmem
Post by: lainz on April 06, 2019, 11:49:48 pm
Ok. I just noticed that in a project I work it was beign used, but not for darwin target for some reason.

Just curiosity.
Title: Re: Lazarus IDE with Cmem
Post by: Thaddy on April 07, 2019, 09:57:01 am
Btw: you can make the use of cmem conditional, so you can first debug with heaptr options and switch to cmem automatically without the heaptrc options like so:
Code: Pascal  [Select][+][-]
  1.  // this is really all you need.Nothing more
  2.  {$if not declared(UseHeapTrace)}cmem,{$ifend}
That enables you to first debug for leaks and subsequently compile for another memory manager.
but as Marco implies: cmem is pretty pointless for Lazarus itself. And cmem is less than optimal with many small allocations with most if not all C flavors.
Title: Re: Lazarus IDE with Cmem
Post by: antispam88 on August 15, 2019, 09:30:54 am
Hi,

I tested
Code: Pascal  [Select][+][-]
  1. {$if not declared(UseHeapTrace)}cmem,{$ifend}

but got an error compiling in Debug mode
Code: Pascal  [Select][+][-]
  1. Project1.pas(6,7) Error: Internal error 200501152

Changing to Release mode, compiling is working, but cmen is still shown as deactivated in IDE.

Best regards,
antispam88

Lazarus: 2.0.4/x86_64-win64-win32/win64
FPC: 3.0.4
Windows: 10 pro. 64
Title: Re: Lazarus IDE with Cmem
Post by: Thaddy on August 15, 2019, 09:54:20 am
That internal error is in symtable.pas and I can not see how this relates to cmem. Can it be something else?
Under fp text mode ide debugging works properly.
You should report the internal error on the bug tracker, preferably with a small and compilable example that reproduces the bug.

I can not reproduce it with a small example like this:
Code: Pascal  [Select][+][-]
  1. program testcmem ;
  2. {$if not declared(useheaptrc)}uses cmem;{$ifend}
  3. type
  4.   TMyrec=record
  5.   a:byte;
  6.   b:integer;
  7.   end;
  8.   PMyRec=^TMyrec;
  9. var
  10.   P:array[0..99] of PMyRec;
  11.   i:integer;
  12. begin
  13.   //allocate some memory
  14.   for i := 0 to 99 do New(P[i]);
  15.   // de-allocate
  16.   for i := 99 downto 0 do Dispose(P[i]);
  17. end.

Make sure you de-selected "use heaptrc" in the Lazarus ide.   

Title: Re: Lazarus IDE with Cmem
Post by: antispam88 on August 15, 2019, 09:56:29 am
sry,

forgot to attach the sample
Title: Re: Lazarus IDE with Cmem
Post by: Thaddy on August 15, 2019, 10:16:34 am
Can not reproduce. Make sure use heaptrc is de-selected in Lazarus
Title: Re: Lazarus IDE with Cmem
Post by: julkas on August 15, 2019, 10:22:44 am
Quote from: marcov on April 06, 2019, 11:37:59 pm
Cmem is not about faster.
Are you shure?
https://benchmarksgame-team.pages.debian.net/benchmarksgame/program/binarytrees-fpascal-7.html
https://forum.lazarus.freepascal.org/index.php?topic=39935.0
Quote
But when using cmem, the code runs approximately 8-10 x faster than the original.

Title: Re: Lazarus IDE with Cmem
Post by: antispam88 on August 15, 2019, 10:31:09 am
Quote from: Thaddy on August 15, 2019, 10:16:34 am
Can not reproduce. Make sure use heaptrc is de-selected in Lazarus

Does this mean, I should not use the "Use Heaptrc unit ..." combobox in project options? I have several build modes and for each one a copy with debugging + heaptrc enabled. 
How is the prefered way to activate heaptrc?

Thanks,
antispam88
Title: Re: Lazarus IDE with Cmem
Post by: julkas on August 15, 2019, 11:27:00 am
Quote from: antispam88 on August 15, 2019, 10:31:09 am
Quote from: Thaddy on August 15, 2019, 10:16:34 am
Can not reproduce. Make sure use heaptrc is de-selected in Lazarus

Does this mean, I should not use the "Use Heaptrc unit ..." combobox in project options? I have several build modes and for each one a copy with debugging + heaptrc enabled. 
How is the prefered way to activate heaptrc?

Thanks,
antispam88
I have tested your project on Win32 Vista, Lazarus 2.0.0 / FPC 3.0.4 both Debug (with heaptrc enabled), Release modes with folowing uses statement.
Code: Pascal  [Select][+][-]
  1. uses
  2.   cmem,
  3.   {$IFDEF UNIX}{$IFDEF UseCThreads}
  4.   cthreads,
  5.   {$ENDIF}{$ENDIF}
  6.   Interfaces, // this includes the LCL widgetset
  7.   Forms, Unit1
  8.   { you can add units after this };

Title: Re: Lazarus IDE with Cmem
Post by: Thaddy on August 15, 2019, 12:24:41 pm
Quote from: antispam88 on August 15, 2019, 10:31:09 am
Quote from: Thaddy on August 15, 2019, 10:16:34 am
Can not reproduce. Make sure use heaptrc is de-selected in Lazarus

Does this mean, I should not use the "Use Heaptrc unit ..." combobox in project options? I have several build modes and for each one a copy with debugging + heaptrc enabled. 
How is the prefered way to activate heaptrc?

Thanks,
antispam88

As I explained on the wiki you can not mix heaptrc with cmem.
https://wiki.freepascal.org/heaptrc

The wiki explains all available options to keep your sources clean.

(Note someone on the forum created a kind of hybrid between heaptrc and cmem, but that is insufficiently tested and the tests I have done do indicate that the patch is unstable or doesn't work at all.)
(Also note that cmem is really only necessary if your program interfaces with C code that does memory allocations. Any other advantages, including speed, are by accident or spurious. I.E you are fooling yourself)
Title: Re: Lazarus IDE with Cmem
Post by: antispam88 on August 15, 2019, 12:50:05 pm
Quote from: Thaddy on August 15, 2019, 12:24:41 pm
As I explained on the wiki you can not mix heaptrc with cmem.
https://wiki.freepascal.org/heaptrc

The wiki explains all available options to keep your sources clean.

(Note someone on the forum created a kind of hybrid between heaptrc and cmem, but that is insufficiently tested and the tests I have done do indicate that the patch is unstable or doesn't work at all.)
(Also note that cmem is really only necessary if your program interfaces with C code that does memory allocations. Any other advantages, including speed, are by accident or spurious. I.E you are fooling yourself)

For that I want to use your code snippet
Code: Pascal  [Select][+][-]
  1. {$if not declared(UseHeapTrace)}cmem,{$ifend} //If heaptrc is activated in project options I don't want to use cmen

But in my environment (laz 2.0.4, fpc 3.0.4, x86_64, win 10 pro 64) I'm getting the compiler error
Code: Pascal  [Select][+][-]
  1. Project1.pas(6,7) Error: Internal error 200501152

Have you tried my example?

Best regards,
antispam88
Title: Re: Lazarus IDE with Cmem
Post by: julkas on August 15, 2019, 12:56:08 pm
OK. Who can give clear explanation - why https://benchmarksgame-team.pages.debian.net/benchmarksgame/program/binarytrees-fpascal-7.html benchmark uses CMem?
Title: Re: Lazarus IDE with Cmem
Post by: marcov on August 15, 2019, 01:09:54 pm
That code runs on Linux, where it is somewhat more standard.

Cmem on Windows is ambiguous by default, since MSVC is the system C compiler, but FPC is mostly compatible with GCC (which again comes in many flavours like cygwin and mingw on Windows). And with many C compiler flavours come many C compiler runtimes.
Title: Re: Lazarus IDE with Cmem
Post by: julkas on August 15, 2019, 01:19:39 pm
Quote
That code runs on Linux, where it is somewhat more standard.
That code runs not only on Linux
Code: Pascal  [Select][+][-]
  1. {$IFDEF UNIX}CThreads,{$ENDIF}
I have tested that code on Win32 Vista.
Title: Re: Lazarus IDE with Cmem
Post by: Thaddy on August 15, 2019, 01:30:41 pm
Yes, but the point is the actual memory aloc/de-alloc implementation of the C compiler you used.
E.g. with other such "tests" on Raspbian and Debian arm I get much differing results between cmem and the default memory manager with GNU C for armhf.
cmem runs out of memory sooner because of the page size. Other C compilers for ARM do have a noticable effect (the one from ARM itself is a good example).
It depends on application., likely even the type of C libraries you interface with if it has any positive impact at all.
Also note the FPC default settings are extremely conservative compared to C compilers, so you may want to explore the optimization options first:
Alignment, CPU settings, optimization -O settings and even the FPU, e.g. vector math for moves is not used in default fpc. But all these things may have been done in a C version for you and they are not done for you in FPC.
Also note the LLVM back-end can improve things. And furthermore, at least on Intel, there are third-party open source Delphi memory managers that outperform both cmem and the fpc default by several magnitudes in a mutithreading application. These memory managers are easily adapted to Freepascal. There are several real - solid - performance tests available for those memory managers in which cmem is also tested and comes second from last. Memory use can be explosive with cmem, whereas fragmentation is an issue with the FPC default, both when stressed.
The delphi ones I mentioned all have per thread allocators, btw. Cmem does not do that, at least not with the GCC compiler.

One thing to note: in the compiler shoot-out (even with an older fpc) the fpc default memory allocator performs best in memory use offset to others that favor speed at the cost of memory.
But then again: memory managers are pluggable in FPC.
Title: Re: Lazarus IDE with Cmem
Post by: marcov on August 15, 2019, 02:45:53 pm
Quote from: julkas on August 15, 2019, 01:19:39 pm
Quote
That code runs on Linux, where it is somewhat more standard.
That code runs not only on Linux
Code: Pascal  [Select][+][-]
  1. {$IFDEF UNIX}CThreads,{$ENDIF}
I have tested that code on Win32 Vista.

That is part of a standard new program template. Not necessarily a sign it was prepared to run on windows.
Title: Re: Lazarus IDE with Cmem
Post by: Thaddy on August 16, 2019, 09:40:24 pm
And the answer to OP is: Yes I did tried your example and gave you the answer.
TinyPortal © 2005-2018