memavail

[Nitorami]

If your memavail works, why does this code run out of memory ?

Code: Pascal  [Select][+][-]

1. var P1,P2: array of byte;
2.     size, i: integer;
3. begin
4.   size := MemAvail;
5.   writeln (size);
6.   SetLength (P1,size div 2);
7.  
8.   size := MemAvail;
9.   writeln (size);
10.   SetLength (P2,size div 2);
11.  
12. end.

[Thaddy]

Because there are too many mistakes in your code.....

Code: Text  [Select][+][-]

1. var P1,P2:packed array of byte;
2.     size, i: PtrUint;
3. begin
4.   size := MemAvail;
5.   writeln (size);
6.   SetLength (P1,size div SizeOf(PtrUint));
7.  
8.   size := MemAvail;
9.   writeln (size);
10.   SetLength (P2,size div SizeOf(PtrUint)));
11.  
12. end.

Untested.... But the div 2 is plainly wrong, as well as the alignment on some platforms. Use the native types and use unsigned types. Also note writeln itself allocates memory (about 732 bytes+data) depending on platform..
Furthermore, allow for the slack of the memory manager.
I guess that if you substract something like 1-4Kb it will work.
Also note  setlength may require a re-allocation, so you demand a theoretical twice as large memory.
This is silly code <grumpy  >

@Nitorami:
Your code is a recipe for disaster when trying to find the limits of your usable memory! <sigh>

[Nitorami]

Thaddy

You misunderstood my point: FPC has no memavail function, because  according to the documentation, prog.pdf, section 8.7 "On modern operating systems, the idea of "Available Free Memory" is not valid for an application."

The OP still posted a custom memavail function.

My code tries to reserve half the memory reported by this function. When doing this several times, we should expect no problem because each time only half the remaining memory is allocated. In fact however this memavail function always reports a constant value, regardless how much memory has already been allocated. Therefore, on the second attempt to reserve "half the available" memory, we get an out of memory exception.

[Thaddy]

No, I did not misunderstood. You still have to allow for the overhead of allocated buffer processing and the slack of the memory manager and the behavior of setlength.....

A memavail just tells you the exact amount of bytes to play with. Given the above limitations.

[Nitorami]

Run the code.
Memavail reports 1.8 GB free
I allocate 50% of that = 900 MB. I can see in the windows task manager that the memory consumption goes up.
Memavail still reports 1.8 GB free. <- THAT is the issue.

[Thaddy]

And your program is compiled with the correct {$SetPEFlags $20} ? 

[marcov]

Run the code.
Memavail reports 1.8 GB free
I allocate 50% of that = 900 MB. I can see in the windows task manager that the memory consumption goes up.
Memavail still reports 1.8 GB free. <- THAT is the issue.

That remains the core problem. No program is in control of all the memory and thus doesn't get the complete overview in the first place. All number are virtualized. Note that memavail is the wrong one anyway, since maxavail returns the max value that you can allocate (since 500000 4kb  pages are 2GB, which memavail would happily report, but doesn't mean you have a 2GB continous addressspace block to allocate in a 32-bit app).

These arguments are not that dire for 32-bits programs since 1 or two won't typically exhaust a modern PCs memory (well, unless you start firefox). But that it doesn't immediately blow up in your face doesn't make the opposite valid.   (*)

And the decision to not support *avail in FPC was made in times when the average PC had 32-64MB.

Even now it remains a hack/shortcut/laziness with limited longevity to avoid having a proper setting. (e.g .what if sb recompiles for 64-bit in a few years).

(*) Russian Roulette has a 5/6 time of everything being fine either, but few will argue it is a good idea.

[Leledumbo]

@leledumbo
You were discussing "approach" outside the snippet I have shown 
and I can't show that code because I'm not the owner (sam707 is) and because it's not a 20 lines  code.
that said, I'm happy for you if you've been hired by a billion dollar company. No Problem, I'm pretty sure billion dollars craps hurt better, in time HEHEHE
... I start the app, MemAvail reports around 1.1GB free, I then start the app, with Blender rendering sumthing on povray plugin, MemAvail repports 1.1GB, I then start libreoffice and load a big presentation, MemAvail repports 1.1GB, as long as no allocation inside the app, MemAvail is CONSTANT because of the memory virtualization, the app has its own address space, ... SO WTF "multitasking environment where available memory changes in millisecond boundary if not less" billion dollars bummer !

See? You're missing the point that your MemAvail only gives VIRTUAL value. What is returned may not be allocatable, only the OS knows the actual available memory at that very particular time. So your MemAvail is again pointless.

[Nitorami]

One additional observation:

Just for interest, I tried the memavail function from the original post

- on my office PC at windows 7, FPC 3.0.4, 32 bit - as said above, it always delivers the same result, regardless how much memory the program has allocated.

- on my home PC, an AMD A4-5300 with 4GB RAM, windows 10, FPC 3.0.4, 32 bit. For some reason it is awfully slow under windows 10. The GetMem calls are getting slower and slower as the amount of allocated memory increases, and the routine did in fact not terminate within 10 minutes.

I have to agree this is not a good solution.

[piGrimm]

@Nitorami
Sorry I should have named the function MaxAvail, and marcov is right, the result is (under 32bits windows) somehow a constant that repports the maximum Heap size (free memory minus your app size minus stacksize). It does not repport that you can allocate that number of bytes into one only contiguous block.
I think you would have to play with the infos inside the TFpcHeapStatus record, to get others infos like largest block, I didn't do it so far. I delivered the code to my boss, and my boss modified it a little, that's what I know
my coding is not perfect, i'm not sam707
Regards.

[piGrimm]

On Win10/32 (and Win7/32) without  {$setpeflags $20} you have a limit around 1.8 GB with the Flag set, the limit ist 3.75 GB.

On Win10/64 windows have stopped working at 200GB. See the tests ins the thread http://forum.lazarus.freepascal.org/index.php/topic,40351.50.html

My PC have 24GB physical memory

Sir, sam707 told me you should not use $20 only, but a mix of ORed constants that you can find in "coffconsts" unit (fpc sources), because depending on future updates, some windows oses perhaps can become  upset if flags are not correctly set

Code: Pascal  [Select][+][-]

1. //Machine types
2. const
3.   IMAGE_FILE_MACHINE_UNKNOWN         = 0;
4.   IMAGE_FILE_MACHINE_I386            = $014c;  // Intel 386.
5. //Coff header characteristics
6. const
7.   IMAGE_FILE_RELOCS_STRIPPED         = $0001;  // Relocation info stripped from file.
8.   IMAGE_FILE_EXECUTABLE_IMAGE        = $0002;  // File is executable  (i.e. no unresolved externel references).
9.   IMAGE_FILE_LINE_NUMS_STRIPPED      = $0004;  // Line nunbers stripped from file.
10.   IMAGE_FILE_LOCAL_SYMS_STRIPPED     = $0008;  // Local symbols stripped from file.
11.   IMAGE_FILE_AGGRESIVE_WS_TRIM       = $0010;  // Agressively trim working set
12.   IMAGE_FILE_LARGE_ADDRESS_AWARE     = $0020;  // App can handle >2gb addresses
13.   IMAGE_FILE_BYTES_REVERSED_LO       = $0080;  // Bytes of machine word are reversed.
14.   IMAGE_FILE_32BIT_MACHINE           = $0100;  // 32 bit word machine.
15.   IMAGE_FILE_DEBUG_STRIPPED          = $0200;  // Debugging info stripped from file in .DBG file
16.   IMAGE_FILE_REMOVABLE_RUN_FROM_SWAP = $0400;  // If Image is on removable media, copy and run from the swap file.
17.   IMAGE_FILE_NET_RUN_FROM_SWAP       = $0800;  // If Image is on Net, copy and run from the swap file.
18.   IMAGE_FILE_SYSTEM                  = $1000;  // System File.
19.   IMAGE_FILE_DLL                     = $2000;  // File is a DLL.
20.   IMAGE_FILE_UP_SYSTEM_ONLY          = $4000;  // File should only be run on a UP machine
21.   IMAGE_FILE_BYTES_REVERSED_HI       = $8000;  // Bytes of machine word are reversed.
22.  

use $16c instead of $20 (14c or 20)
regards