Recent

Author Topic: Size of memory  (Read 5133 times)

LV

  • Full Member
  • ***
  • Posts: 153
Re: Size of memory
« Reply #60 on: October 25, 2024, 02:19:35 pm »
I had some free time and decided to check on two more systems: Linux(Deb64) and Win32.

Result:

1. Win64, Linux(Deb64)
FreeMem(GetMem(N)) = if N ≦ 536, then MemSize(GetMem(N)) + SizeOf(Pointer);
else MemSize(GetMem(N)) + SizeOf(Pointer) + 16;

2. Win32
FreeMem(GetMem(N)) = if N ≦ 524, then MemSize(GetMem(N)) + SizeOf(Pointer);
else MemSize(GetMem(N)) + SizeOf(Pointer) + 8;
« Last Edit: October 25, 2024, 02:22:20 pm by LV »

Thaddy

  • Hero Member
  • *****
  • Posts: 16138
  • Censorship about opinions does not belong here.
Re: Size of memory
« Reply #61 on: October 25, 2024, 02:25:26 pm »
Yes, the allocator uses a power of two.
I believe I once complained that it should be close to the golden circle (~1.6) instead.
(discussion was with Maciej)
« Last Edit: October 25, 2024, 02:29:00 pm by Thaddy »
If I smell bad code it usually is bad code and that includes my own code.

BrunoK

  • Hero Member
  • *****
  • Posts: 623
  • Retired programmer
Re: Size of memory
« Reply #62 on: October 25, 2024, 05:03:15 pm »
Why did I get  interested in the subject ? Because when using cmem, MemSize returns exactly what was requested

(but that might be because cmem can't look inside the libc allocator. It can only works through its abstraction)
cmem CAN look allocated size in most of the malloc libraries used. (See numerous discussions with Grumpy)
That, I tested against windows malloc and WinHeap. The funny thing is that only Windows heap announces the HeapSize function as supported. Other implementations are very vague as to it being  durably supported.

BUT reaching the MemSize in any of the malloc's is a very long path if you trace the assembler whereas cmem stores it at a very quickly accessible negative offset relative to the getmem pointer. That's what is useful for AnsiStrings.

The only thing that might improve speed when using cmem would be to round up the memory request .

My customized FPC 3.2.2 has in .\3.2.2-git\rtl\inc\astrings.inc
Code: Pascal  [Select][+][-]
  1. const
  2.   RoundUp  = SizeUInt(8);     // Chunk up to 8 bytes block
  3.   Rounding = SizeUInt(not 8); // Mask size
  4.   AnsiUpWithOffset = AnsiFirstOff + SizeOf(char) + RoundUp;
  5.  
  6. {$ifndef FPC_HAS_NEWANSISTR}
  7. {$define FPC_HAS_NEWANSISTR}
  8. Function NewAnsiString(Len : SizeInt) : Pointer;
  9. {
  10.   Allocate a new AnsiString on the heap.
  11.   initialize it to zero length and reference count 1.
  12. }
  13. Var
  14.   P : Pointer;
  15. begin
  16.   { request a multiple of 16 because the heap manager alloctes anyways chunks of 16 bytes }
  17.   // GetMem(P,Len+(AnsiFirstOff+sizeof(char)));
  18.   GetMem(P, (Len+AnsiUpWithOffset) and Rounding);
  19.   If P<>Nil then
  20.  
That may (maybe not ...) help a bit cmem usage by retarding need to grab a bigger block for a reallocated string, except malloc when asked more memory for a extended reallocation will first check if the allocated chunck is large enough.

LV

  • Full Member
  • ***
  • Posts: 153
Re: Size of memory
« Reply #63 on: October 26, 2024, 03:21:23 pm »
Let's build an аlgorithm for calculating the total allocated memory size without calling memory manager functions. After sitting for a while in my leisure time, something seems to have been resolved. It has been tested on systems: Win32, Win64, and Linux(Deb64).

Code: Pascal  [Select][+][-]
  1. Program project1;
  2.  
  3. const
  4.   BaseChunk = 512;
  5.  
  6. Var
  7.  P : Pointer;
  8.  MGet, MTotalAloc : Longint;
  9.  i, Scale, S2, S3, S4, S5, S8 : Integer;
  10.  F: PtrUInt;
  11.  
  12. begin
  13.   Scale := SizeOf(Pointer);
  14.   writeln('Your system is ', Scale * 8,' bit');
  15.   // or enter the system manually: for 64bit Scale := 8
  16.   // and for 32bit Scale := 4
  17.   S2 := Scale * 2;
  18.   S3 := Scale * 3;
  19.   S4 := Scale * 4;
  20.   S5 := Scale * 5;
  21.   S8 := Scale * 8;
  22.  
  23.   for i := 0 to 1024 do
  24.   begin
  25.     MGet := i + 1;
  26.  
  27.     // Algorithm for calculating the total allocated memory size
  28.     // (without calling memory manager functions)
  29.     if MGet <= S3 then
  30.       MTotalAloc := S4
  31.     else
  32.       if (MGet <= BaseChunk + S3) and (MGet > S3) then
  33.       MTotalAloc := S8 + S4 * ((MGet - (S3 + 1)) div S4)
  34.     else
  35.       if (MGet <= BaseChunk + S5) and (MGet > (BaseChunk + S3)) then
  36.       MTotalAloc := S8 + S4 * (BaseChunk div S4) +
  37.         S2 * ((MGet - (BaseChunk + S3 + 1)) div S2)
  38.     else
  39.       MTotalAloc := S8 + S4 * (BaseChunk div S4) + S4 +
  40.         S4 * ((MGet - (BaseChunk + S5 + 1)) div S4);
  41.  
  42.  
  43.     write('MGet = ', MGet);
  44.     write('   MTotalAloc = ', MTotalAloc);
  45.  
  46.     // Only to check the correctness of the algorithm
  47.     P := GetMem(MGet);
  48.     F := FreeMem(P);
  49.     write('   FreeMem = ', F);
  50.     writeln('   (MTotalAloc - FreeMem) = ', MTotalAloc - F);
  51.     if (MGet = S3) or (MGet = BaseChunk + S3) or (MGet = BaseChunk + S5) then
  52.       writeln('-------------------------------------------------------');
  53.   end;
  54.  
  55.   readln;
  56. end.  
  57.  

Maybe this is the answer to one of the questions.

Here is an issue about this topic: https://gitlab.com/freepascal.org/fpc/documentation/-/issues/39403

LV

  • Full Member
  • ***
  • Posts: 153
Re: Size of memory
« Reply #64 on: October 27, 2024, 07:47:50 am »
To summarize my attempts to answer the questions on this topic here is a program that includes the total memory allocation algorithm and calls to the memory manager functions. "Please do not shoot the piano player; he is doing his best" :)
Code: Pascal  [Select][+][-]
  1. program project1;
  2.  
  3. const
  4.   BaseChunk = 512;
  5.  
  6. var
  7.   P: Pointer;
  8.   MGet, MGetSize, MGetSizeScale, MTAlloc: longint;
  9.   i, Scale, S2, S3, S4, S5, S8: integer;
  10.   F: PtrUInt;
  11.  
  12. begin
  13.   Scale := SizeOf(Pointer);
  14.   writeln('Your system is ', Scale * 8, ' bit');
  15.   // or enter the system manually: for 64bit Scale := 8
  16.   // and for 32bit Scale := 4
  17.   S2 := Scale * 2;
  18.   S3 := Scale * 3;
  19.   S4 := Scale * 4;
  20.   S5 := Scale * 5;
  21.   S8 := Scale * 8;
  22.  
  23.   for i := 0 to 1024 do
  24.   begin
  25.  
  26.     MGet := i + 1;
  27.  
  28.     // Algorithm for calculating the total allocated memory size
  29.     // (without calling memory manager functions)
  30.     if MGet <= S3 then
  31.       MTAlloc := S4
  32.     else
  33.     if (MGet <= BaseChunk + S3) and (MGet > S3) then
  34.       MTAlloc := S8 + S4 * ((MGet - (S3 + 1)) div S4)
  35.     else
  36.     if (MGet <= BaseChunk + S5) and (MGet > (BaseChunk + S3)) then
  37.       MTAlloc := S8 + S4 * (BaseChunk div S4) + S2 *
  38.         ((MGet - (BaseChunk + S3 + 1)) div S2)
  39.     else
  40.       MTAlloc := S8 + S4 * (BaseChunk div S4) + S4 + S4 *
  41.         ((MGet - (BaseChunk + S5 + 1)) div S4);
  42.  
  43.     // Using memory manager functions
  44.     P := GetMem(MGet);
  45.     if MGet <=(BaseChunk + S3) then
  46.     begin
  47.       MGetSize := MemSize(P);
  48.       MGetSizeScale := MemSize(P) + Scale;
  49.     end
  50.     else
  51.     begin
  52.       MGetSize := MemSize(P);
  53.       MGetSizeScale := MemSize(P) + Scale * 3;
  54.     end;
  55.  
  56.     F := FreeMem(P);
  57.  
  58.  
  59.     Write('MGet = ', MGet);
  60.     Write('   MGetSize = ', MGetSize, '||');
  61.     Write('   MGetSizeScale = ', MGetSizeScale);
  62.     Write('   FreeMem = ', F);
  63.     Write('   | MTAlloc = ', MTAlloc, ' |');
  64.     writeln('   Error = ', MTAlloc - F);
  65.     if (MGet = S3) or (MGet = BaseChunk + S3) or (MGet = BaseChunk + S5) then
  66.       writeln('-------------------------------------------------------');
  67.   end;
  68.  
  69.   readln;
  70. end.  
  71.  

Best regards

LV

  • Full Member
  • ***
  • Posts: 153
Re: Size of memory
« Reply #65 on: November 02, 2024, 10:44:17 am »
Hi,

Can I ask why result of MemSize is different of what I asked from GetMem?
And is the memory of that size is safe to use?
Code: Pascal  [Select][+][-]
  1. WriteLn(MemSize(GetMem(1024)));
This code returns 1032 on Windows instead of 1024. Why? And can I use those 1032 bytes safely?

Also, what is the difference between MemSize and SysMemSize? I can not find anything about that in the documentation?
I can not find anything about SysFreememSize too, what it does?

There is no bug if you set
Project Options -> Compiler Options -> Debugging -> Use Heaptrc unit (check for mem-leaks) (-gh)

Code: Pascal  [Select][+][-]
  1. program project1;
  2.  
  3. var
  4.   P: Pointer;
  5.   M: ptruint;
  6.  
  7. begin
  8.  
  9.   GetMem(P, 1024);
  10.   writeln('GetMem   ', memsize(P));
  11.   M := MemSize(P);
  12.   writeln('MemSize  ', M);
  13.   FreeMem(P, 1024);
  14.   writeln('FreeMem  ', memsize(P));
  15.  
  16.   readln;
  17.  
  18. end.
  19.  

Output for Win32, Win64, Linux(Deb64)
Code: Text  [Select][+][-]
  1. GetMem   1024
  2. MemSize  1024
  3. FreeMem  1024
  4.  

Can anyone confirm this?

BrunoK

  • Hero Member
  • *****
  • Posts: 623
  • Retired programmer
Re: Size of memory
« Reply #66 on: November 02, 2024, 10:58:21 am »
There is no bug if you set
Project Options -> Compiler Options -> Debugging -> Use Heaptrc unit (check for mem-leaks) (-gh)
Heaptrc reports its internal Memsize allocation. This does not reflect effectively used memory supplied by heap.inc.

LV

  • Full Member
  • ***
  • Posts: 153
Re: Size of memory
« Reply #67 on: November 02, 2024, 11:01:35 am »
Thank you

jamie

  • Hero Member
  • *****
  • Posts: 6733
Re: Size of memory
« Reply #68 on: November 02, 2024, 06:23:11 pm »
Code: Pascal  [Select][+][-]
  1.  
  2.     //Sanity Checks For Vertices Size vs. Count
  3.     if MemSizeLessThan(MemSize(Vertices), PtrUInt(SizeOf(tagTRIVERTEX)*NumVertices)) then
  4.       exit;
  5.  
  6.     //Sanity Checks For Meshes Size vs. Count
  7.     if MemSizeLessThan(MemSize(Meshes), PtrUInt(MeshSize[DoFillTriangle]*NumMeshes)) then
  8.       exit;                                                                                      
  9.  
  10.  

So, I guess this is nothing to be alarmed about?

Found in most of the Widget's files for the Gradient fill code etc.
The only true wisdom is knowing you know nothing

LV

  • Full Member
  • ***
  • Posts: 153
Re: Size of memory
« Reply #69 on: November 02, 2024, 09:34:14 pm »
Thank you. The [Use Heaptrc unit (check for mem-leaks) (-gh)] option is producing the correct results. Honestly, I don’t have the desire or time to delve into the details of heap.inc. This was more of a matter of curiosity than a practical necessity. :)

 

TinyPortal © 2005-2018