Recent

Author Topic: memavail  (Read 9375 times)

piGrimm

  • Guest
memavail
« on: June 04, 2018, 10:19:49 pm »
trick to get that old COOL MemAvail function like in turbo pascal
(tested on Lazarus 1.8.5 fixes, 32 bits windows... should work on any platform tho)
Please check, discuss, improve the code, share, thanks!
function returns the available largest block of ram actually available to the FPC MemoryManager SO do NOT compile it while using cmem unit

Code: Pascal  [Select][+][-]
  1. function MemAvail: PtrUInt;
  2. var
  3.   h: TFPCHeapStatus;
  4.   i: cardinal;
  5.   LastHeapFails: boolean;
  6.   Z: Pointer;
  7. begin
  8.   Result := 0;
  9.   LastHeapFails := ReturnNilIfGrowHeapFails;
  10.   ReturnNilIfGrowHeapFails := True;
  11.   for i := 1 to $FFFF do
  12.   begin
  13.     Z := GetMem(i * $10000);
  14.     if Z = nil then
  15.       break;
  16.     h := GetFPCHeapStatus;
  17.     Result := h.MaxHeapSize;
  18.     Freemem(Z);
  19.   end;
  20.   ReturnNilIfGrowHeapFails := LastHeapFails;
  21. end;
  22.  

piGrimm

  • Guest
Re: memavail
« Reply #1 on: June 04, 2018, 10:24:39 pm »
at 1st I took $4000 (16K block increment) but the loop can become long on machines with large amount of ram, so I modified to $1000 (64K block) but the final result is less precise, I guess there should be other loops with variable increments to get EXACT Mem Avail

marcov

  • Administrator
  • Hero Member
  • *
  • Posts: 12706
  • FPC developer.
Re: memavail
« Reply #2 on: June 04, 2018, 10:26:08 pm »
discuss, improve the code, share, thanks!

Code: Pascal  [Select][+][-]
  1. function MemAvail: PtrUInt;
  2. var
  3.   h: TFPCHeapStatus;
  4.   i: cardinal;
  5.   LastHeapFails: boolean;
  6.   Z: Pointer;
  7. begin
  8.   Result := 0;
  9.   LastHeapFails := ReturnNilIfGrowHeapFails;
  10.   ReturnNilIfGrowHeapFails := True;
  11.   for i := 1 to $FFFF do
  12.   begin
  13.     Z := GetMem(i * $10000);
  14.     if Z = nil then       // here the system memory situation  can already be different
  15.       break;
  16.     h := GetFPCHeapStatus; // here the system memory situation can be  AGAIN different
  17.     Result := h.MaxHeapSize;  // this is also again totally different again
  18.     Freemem(Z);
  19.   end;
  20.   ReturnNilIfGrowHeapFails := LastHeapFails;
  21. end;
  22.  

Which makes it effectively a loop around 3 race conditions.

Memavail is a bad idea since applications share RAM (read: windows 3.x), and the current running application can't assume that all the memory is for himself. 

Applications will eat all memory this way, and the first next application will begin to swap (even though the first app needs some of his data only once per day).

Best is to simply make the size of your caches or whatever scalable datastructure you have configurable in some ini, using a cmdline param or, in a GUI app somewhere in your options or configuration dialogue.

This way, an user can strike his own balance depending on his hardware and his demands.
« Last Edit: June 04, 2018, 10:32:17 pm by marcov »

piGrimm

  • Guest
Re: memavail
« Reply #3 on: June 04, 2018, 10:30:57 pm »

Memavail is a bad idea since applications share RAM (read: windows 3.x), and the current running application can't assume that all the memory is for himself.

I know ram is a pool to the other apps, BUT I need to ensure MY app have enough at some pointS, so a MemAvail function called sometimes IS A GOOD IDEA

marcov

  • Administrator
  • Hero Member
  • *
  • Posts: 12706
  • FPC developer.
Re: memavail
« Reply #4 on: June 04, 2018, 10:33:28 pm »
I know ram is a pool to the other apps, BUT I need to ensure MY app have enough at some pointS, so a MemAvail function called sometimes IS A GOOD IDEA

No, you just allocate what you need. If it fails, you can't ensure it. 

And as said if what you need is dynamic depending on use or datasize, you can make it a setting somewhere.

piGrimm

  • Guest
Re: memavail
« Reply #5 on: June 04, 2018, 10:39:29 pm »
NO
Example :
I load pictures from a folder in a loop and if MemAvail goes under 512K I messagebox my user before an exception occurs, actually it works, avoiding future SIGSEGV or whatever ENOMEM try except block

piGrimm

  • Guest
Re: memavail
« Reply #6 on: June 04, 2018, 10:40:32 pm »
I don't like try except blocks, I use them only when i can not do the job the easy way

Leledumbo

  • Hero Member
  • *****
  • Posts: 8835
  • Programming + Glam Metal + Tae Kwon Do = Me
Re: memavail
« Reply #7 on: June 05, 2018, 12:19:03 am »
I don't like try except blocks, I use them only when i can not do the job the easy way
But your easy way is wrong. Your approach may be working, but it's a bad UX. First, as Marco said, your "checking" code may push other apps into the swap unwantedly. Second, your app might actually be able to allocate that much memory a split second right after your "checking" code says it's not possible. Exceptions are invented exactly for situations like this, where you cannot ensure something is guaranteed to work but yet it's still worth trying.

Don't just because you don't like a concept, you take the wrong direction to solve a problem.

piGrimm

  • Guest
Re: memavail
« Reply #8 on: June 05, 2018, 01:22:21 am »
I don't like try except blocks, I use them only when i can not do the job the easy way
But your easy way is wrong. Your approach may be working, but it's a bad UX. First, as Marco said, your "checking" code may push other apps into the swap unwantedly. Second, your app might actually be able to allocate that much memory a split second right after your "checking" code says it's not possible. Exceptions are invented exactly for situations like this, where you cannot ensure something is guaranteed to work but yet it's still worth trying.

Don't just because you don't like a concept, you take the wrong direction to solve a problem.

huh?!... just wondering ... how can you tell my approach is wrong without seeing any line of my code which is part of a project from sam707 sold about 40000 times?  that's curious, Master... HAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAH tell me more about your crystal ball  :D
« Last Edit: June 05, 2018, 01:36:06 am by piGrimm »

piGrimm

  • Guest
Re: memavail
« Reply #9 on: June 05, 2018, 01:28:42 am »
approach 1
1) check mem
2) if limit arise, give choice to user BEFORE
3) continue or not
approach 2 using exceptions
1) go blind load pictures
2) AFTER mem crash try something in the face of the user
3) close the app on game over
Ok prefer 2 LOL cuz MazTer said HAHAHAHAH, I no need a 'prevent' comportment = I damn prefer a crash
« Last Edit: June 05, 2018, 01:30:30 am by piGrimm »

engkin

  • Hero Member
  • *****
  • Posts: 3112
Re: memavail
« Reply #10 on: June 05, 2018, 02:02:22 am »
@piGrimm,

As an improvement, here is a faster way using binary search. Notice that regardless of the memory manager used, you can get the precise* available memory from Lo-1:
Code: Pascal  [Select][+][-]
  1. function MemAvail: PtrUInt;
  2. var
  3.   h: TFPCHeapStatus;
  4.   LastHeapFails: boolean;
  5.   Z: Pointer;
  6.   Lo, Hi, M: PtrUInt;
  7. begin
  8.   Result := 0;
  9.   LastHeapFails := ReturnNilIfGrowHeapFails;
  10.   ReturnNilIfGrowHeapFails := True;
  11.  
  12.   { Use binary search }
  13.   Lo := 1;  Hi := High(PtrUInt);
  14.   while Lo<=Hi do
  15.   begin
  16.     M := Lo + (Hi - Lo) div 2;
  17.     Z := GetMem(M);
  18.     if Z <> nil then
  19.       Lo := M+1
  20.     else
  21.       Hi := M-1;
  22.  
  23.     h := GetFPCHeapStatus;
  24.     Result := h.MaxHeapSize;
  25.     Freemem(Z);
  26.   end;
  27.  
  28.   ReturnNilIfGrowHeapFails := LastHeapFails;
  29. end;


Unfortunately I do agree with Marco and Leledumbo. :-(


* As long nothing reserves memory before you do.

Edit:
Code is *not* tested.
« Last Edit: June 05, 2018, 02:03:59 am by engkin »

ASerge

  • Hero Member
  • *****
  • Posts: 2477
Re: memavail
« Reply #11 on: June 05, 2018, 05:53:09 am »
This
Please check, discuss, improve the code, share, thanks!
and this
huh?!... just wondering ... how can you tell my approach is wrong without seeing any line of my code which is part of a project from sam707 sold about 40000 times?  that's curious, Master... HAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAH tell me more about your crystal ball  :D
You want to discuss, so I say your approach is really WRONG!
How many people have to tell you that it's wrong until you accept?

Leledumbo

  • Hero Member
  • *****
  • Posts: 8835
  • Programming + Glam Metal + Tae Kwon Do = Me
Re: memavail
« Reply #12 on: June 05, 2018, 06:38:55 am »
huh?!... just wondering ... how can you tell my approach is wrong without seeing any line of my code which is part of a project from sam707 sold about 40000 times?  that's curious, Master... HAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAH tell me more about your crystal ball  :D
You posted the code, of course I saw it. Even if you don't by the way you wrote and described it I can tell how wrong that is. We're not living in single tasking OS world anymore where you can precisely check currently available memory. In TP that is used to check whether available memory is enough, if yes follow by really allocating that much. That WON'T work in multitasking environment where available memory changes in millisecond boundary if not less.
Not even a code that's been sold a billion times guarantee any correctness. Even the likeliness of the code using emergency approach is bigger due to time constraints vs unpaid one. Been there, done that in a company that sells billion dollars product sold for years.

piGrimm

  • Guest
Re: memavail
« Reply #13 on: June 05, 2018, 09:53:33 am »
@leledumbo
You were discussing "approach" outside the snippet I have shown  :P
and I can't show that code because I'm not the owner (sam707 is) and because it's not a 20 lines  :D code.
that said, I'm happy for you if you've been hired by a billion dollar company. No Problem, I'm pretty sure billion dollars craps hurt better, in time HEHEHE
... I start the app, MemAvail reports around 1.1GB free, I then start the app, with Blender rendering sumthing on povray plugin, MemAvail repports 1.1GB, I then start libreoffice and load a big presentation, MemAvail repports 1.1GB, as long as no allocation inside the app, MemAvail is CONSTANT because of the memory virtualization, the app has its own address space, ... SO WTF "multitasking environment where available memory changes in millisecond boundary if not less" billion dollars bummer !
« Last Edit: June 05, 2018, 10:13:09 am by piGrimm »

af0815

  • Hero Member
  • *****
  • Posts: 1409
Re: memavail
« Reply #14 on: June 05, 2018, 10:26:32 am »
On Win10/32 (and Win7/32) without  {$setpeflags $20} you have a limit around 1.8 GB with the Flag set, the limit ist 3.75 GB.

On Win10/64 windows have stopped working at 200GB. See the tests ins the thread http://forum.lazarus.freepascal.org/index.php/topic,40351.50.html

My PC have 24GB physical memory
regards
Andreas

 

TinyPortal © 2005-2018