Forum > Beginners

Memory layout and management questions

(1/3) > >>

K.IO.s:
Hi there, i'd like to know more about when the stack i used for memory and when the heap is used.

In C i'd assume that anything that wasn't malloc'd goes into the stack, but it seems that is not the case with freePascal.
One of my main doubts was about arrays, it seems that if you declare them at the file level(global), they will be allocated on the heap, is there any way to force them to be allocated on the stack?
The same question with regards to shortStrings, or fixed strings, normally this type of thing is bundled in the executable, is that not the case with pascal?

Another issue is I was reading about "dynamic arrays", and specially in the case of 'arrays of arrays', it seems that they don't have a contiguous memory layout, and thus the data is allocated all over the place with pointers everywhere.
 In c++ the vector class is a dynamic array but it does maintain the linearity of it's layout, so you can access it as any regular fixed array. I guess there is no equivalent in freePascal.

So for performance oriented tasks, I'm still not sure if I should use an array[a,b], or a 1d array, and use the old C style syntax to access values as if they were in a 2d space.(as an example). This is for trying to avoid cache misses.

I'm also trying to figure out how to handle the creation of custom allocators in pascal.
It seems that I need to look into the memory manager, but I haven't understood it completely yet. I'm not sure if I can have multiple allocation strategies in the same program, or only one global handling for everything.(which would defeat much of the purpose for custom allocators)

If there is anything else i should know about the memory aspect of the language, please let me know.
And feel free to correct me with regards to anything, still trying to figure this out.

Thanks either way.

440bx:

--- Quote from: K.IO.s on July 05, 2022, 10:45:42 pm ---In C i'd assume that anything that wasn't malloc'd goes into the stack, but it seems that is not the case with freePascal.

--- End quote ---
In some instances, FreePascal somewhat hides where memory is allocated.  For instance, classes are always allocated on the heap and that's not obvious just reading the code.


--- Quote from: K.IO.s on July 05, 2022, 10:45:42 pm ---One of my main doubts was about arrays, it seems that if you declare them at the file level(global), they will be allocated on the heap, is there any way to force them to be allocated on the stack?

--- End quote ---
if you declare and array or any variable of any type for that matter at the _global_ level then it will be allocated in the program's data segment. In that case, it is neither on the heap nor the stack.  It's important to realize that the location _where_ a variable is declared affects where it will be located (data segment, stack, heap) and, that in a significant number of cases the actual allocation requires "programmer intervention" (usually for dynamically sized structures/variables.) Just in case, no heap allocation depends on where the variable is declared.  Heap allocations are always "manual" (even though the mechanism may be hidden.)


--- Quote from: K.IO.s on July 05, 2022, 10:45:42 pm ---...it seems that they don't have a contiguous memory layout, and thus the data is allocated all over the place with pointers everywhere.

--- End quote ---
No, the arrays the compiler builds always have contiguous elements, the compiler depends on that to access the array elements.  IOW, allocation isn't all over the place, linearity is always preserved.


--- Quote from: K.IO.s on July 05, 2022, 10:45:42 pm ---So for performance oriented tasks, I'm still not sure if I should use an array[a,b], or a 1d array, and use the old C style syntax to access values as if they were in a 2d space.(as an example). This is for trying to avoid cache misses.

--- End quote ---
How an array is accessed is easy to change.  First thing is to get the program to work correctly then, once it works correctly, keep it working equally well but faster.  It's the Toyota way, first do it right, then, keep doing it right but faster.  They proved it's an effective way of getting good things done.



--- Quote from: K.IO.s on July 05, 2022, 10:45:42 pm ---It seems that I need to look into the memory manager, but I haven't understood it completely yet.

--- End quote ---
If you want performance and simplicity and, don't mind some extra work, manage the memory yourself.  You can get very significant performance gains if the custom memory management routines are a good match to the program's behavior.

HTH.

K.IO.s:

--- Quote from: 440bx on July 06, 2022, 12:13:45 am ---If you want performance and simplicity and, don't mind some extra work, manage the memory yourself.  You can get very significant performance gains if the custom memory management routines are a good match to the program's behavior.

HTH.

--- End quote ---

Thanks for the extended reply, awesome.
So basically it works like in C as expected. I read a bunch of posts in the forum that seemed to indicate otherwise. Maybe they are referring to older implementations.


I have a follow up question about the memory manager part. If you are familiar with it, is it possible to get scoped memory management with the MM pascal has, or something of that nature?

In c++17 we had pmr allocators, which allowed for a mix and matching of allocation strategies, i'd be looking for something similar.
Like using a bump allocator in some part of the code, a pool in another and a general freelist for the rest.
I'm still trying to figure out if this can be managed in freePascal.

Thanks again.

K.IO.s:
For the sake of reference, here is one example of posts that I saw that seemed to indicate arrays were not contiguous in freePascal: https://forum.lazarus.freepascal.org/index.php/topic,39739.msg273701.html#msg273701

And here for an example saying arrays are auto allocated on the heap:
https://forum.lazarus.freepascal.org/index.php/topic,34989.msg230535.html#msg230535

There were others where people explicitly stated such things. That's what got me confused in the first place. Why would the language have such obscure design choices.

440bx:

--- Quote from: K.IO.s on July 06, 2022, 12:39:58 am ---Thanks for the extended reply, awesome.

--- End quote ---
You're welcome.


--- Quote from: K.IO.s on July 06, 2022, 12:39:58 am ---So basically it works like in C as expected.

--- End quote ---
It pretty much works as it does in C but, there some occasional differences/peculiarities.  For instance, FPC will always allocate classes on the heap.  In C++ it is possible to control where a class resides in memory.  FP allows a programmer to control where an _object_ (not class but, old object) or advanced record (inheritance-less "object") can reside in memory.



--- Quote from: K.IO.s on July 06, 2022, 12:39:58 am ---I read a bunch of posts in the forum that seemed to indicate otherwise. Maybe they are referring to older implementations.

--- End quote ---
some people have misconceptions about what the compiler does and/or how it does it.  When in doubt, I read the assembly code, that always tells you what reality is.


--- Quote from: K.IO.s on July 06, 2022, 12:39:58 am ---is it possible to get scoped memory management with the MM pascal has, or something of that nature?

--- End quote ---
Local variables are inherently scoped.  They exist only until the function/procedure in which they are declared exits.  Variables (as well as types and functions) declared in a unit's interface are in the global scope (data segment in the case of variables) but, they are only accessible by code that declares they use the unit. IOW, what's scoped is not the existence of the variable but, its visibility.


--- Quote from: K.IO.s on July 06, 2022, 12:39:58 am ---In c++17 we had pmr allocators, which allowed for a mix and matching of allocation strategies, i'd be looking for something similar.

--- End quote ---
I cannot answer that question.  I stay away from all the OOP stuff.  Except in throw away programs, I always do my own memory management.   It's more work but, much more efficient and powerful.


--- Quote from: K.IO.s on July 06, 2022, 12:39:58 am ---I'm still trying to figure out if this can be managed in freePascal.

--- End quote ---
if you're willing to do your own memory management, you can do in FreePascal anything you can do in C.  As far as, what memory management FP can do automatically for you, I cannot provide a good answer because, as I stated before, I deliberately stay away from that stuff. 


--- Quote from: K.IO.s on July 06, 2022, 12:39:58 am ---Thanks again.

--- End quote ---
You're welcome again. :)

Navigation

[0] Message Index

[#] Next page

Go to full version