Lazarus

Free Pascal => General => Topic started by: flowCRANE on December 13, 2024, 03:09:56 pm

Title: [SOLVED] Classic objects — does FPC generate additional code?
Post by: flowCRANE on December 13, 2024, 03:09:56 pm
From what I understand, classic objects, i.e. data structures declared with the object keyword, take up the same amount of memory as their record counterparts. We can declare such objects on the stack and on the heap, but unlike records, objects support inheritance. So they are something between records and classes, which gives additional possibilities, which I am happy to use.

In my game engine, I use such classic objects to create simple data structures that contain only fields with data (no methods) and this data can be inherited. Such objects can be used as local variables existing on the stack, but can also be allocated on the heap. When allocated on the heap, memory for them is always allocated using GetMem and freed using FreeMem. That's it in a nutshell.

Assuming that:


can someone familiar with FPC internals officially confirm that using objects only with fields and not using the API dedicated to classic objects (constructors, destructors, New, Dispose etc.) makes such objects exactly the same as regular records? So far, I have not found that using them in the manner mentioned above would result in the automatic allocation of additional memory or forces the compiler to generate additional instructions.

Thanks in advance for your answers.
Title: Re: Classic objects — does FPC generate additional code?
Post by: marcov on December 13, 2024, 03:18:05 pm
What OO do you have left then?  Records can contain methods. TP objects only allocate a VMT when virtual methods are defined.

I'm not sure using getmen and freemem handle fields with automated types. You would have to add initialize and finalize calls for that.

I would simply use records and use new and dispose.
Title: Re: Classic objects — does FPC generate additional code?
Post by: flowCRANE on December 13, 2024, 03:54:02 pm
What OO do you have left then?  Records can contain methods.

As I wrote, I don't need methods and properties — simple fields only.
I use objects instead of records because objects support inheritance and records do not.

Quote
TP objects only allocate a VMT when virtual methods are defined.

Ok, I'm glad to hear this.

Quote
I'm not sure using getmen and freemem handle fields with automated types. You would have to add initialize and finalize calls for that.

This is not a problem, because I always declare an initialization and finalization function, as well as an allocating and deallocating function for each record and object. This way I can freely use such data structures and allocate them on both the stack and the heap, regardless of whether they contain fields that require initialization/finalization.

Thank you for the answer!
Title: Re: Classic objects — does FPC generate additional code?
Post by: Warfley on December 13, 2024, 03:56:32 pm
I don't think so. In the internal representation in the FPC objects are closer to classes than to records (classes are just represented as objects with a special flag while records have their own internal representation). While it's true that objects do not get a vmt if they don't need one, I don't think they give the same guarantees as records with internal layout. I think like classes, object fields can be reordered by the optimizer.
So generally speaking I would not bet on it
Title: Re: Classic objects — does FPC generate additional code?
Post by: flowCRANE on December 13, 2024, 04:05:15 pm
Guys, I need facts, not beliefs or opinions.

I found no evidence that FPC generates additional code or allocates additional memory for TP objects containing only simple fields. But I'd rather ask and make it clear once and for all. And I'd prefer someone who knows exactly how the compiler handles TP objects to answer. Even on this forum I have found beliefs that TP objects are not compatible with records, although no concrete evidence has been provided to support these theses.

Here I'm asking about a specific case — an TP object with simple fields only (no methods, especially virtual ones).
Title: Re: Classic objects — does FPC generate additional code?
Post by: marcov on December 13, 2024, 04:32:16 pm
What OO do you have left then?  Records can contain methods.

As I wrote, I don't need methods and properties — simple fields only.
I use objects instead of records because objects support inheritance and records do not.

Non virtual inheritance is possible I guess, that wouldn't require a VMT.  But any kind of polymorphism will be hard.
Title: Re: Classic objects — does FPC generate additional code?
Post by: flowCRANE on December 13, 2024, 04:39:40 pm
But any kind of polymorphism will be hard.

It's not a problem, neither for records nor for TP objects. That's because whenever I need to pass a structure reference or TP object, I do it via a weak (untyped) pointer, thanks to {$TYPEDADDRESS OFF} mode. This gives me full polymorphism (for any data) and eliminates the need for the notorious casting.
Title: Re: Classic objects — does FPC generate additional code?
Post by: Thaddy on December 13, 2024, 05:15:51 pm
NO

simple and short answer.
Title: Re: Classic objects — does FPC generate additional code?
Post by: flowCRANE on December 13, 2024, 08:12:59 pm
As I wrote — I need facts, not simple and short answers. Beliefs don't work well with technology.
Title: Re: Classic objects — does FPC generate additional code?
Post by: Warfley on December 13, 2024, 10:46:49 pm
The problem is that you are asking something that is undocumented. The documentation for records clearly state their memory layout and management, records are designed to be ABI stable, objects not.
Maybe what you want works, maybe it doesn't, maybe it works now but not anymore in the next version of the FPC. FPC has a few hundred thousand lines of code, a quick search for the corresponding type tobjectdef shows over 1000 occurrences in the code. So making a definite claim is not really easy

At least from the documentation I can tell you that ABI stability is not a design criterium of objects and as such even if it may be the case now there is no guarantee it will stay this way
Title: Re: Classic objects — does FPC generate additional code?
Post by: flowCRANE on December 13, 2024, 11:18:26 pm
The problem is that you are asking something that is undocumented. […] FPC has a few hundred thousand lines of code, a quick search for the corresponding type tobjectdef shows over 1000 occurrences in the code. So making a definite claim is not really easy.

And that's why I'm asking you guys, hoping to find out something that is not in the documentation. 8)

Quote
At least from the documentation I can tell you that ABI stability is not a design criterium of objects and as such even if it may be the case now there is no guarantee it will stay this way

Now that's a clue.

It's a pity that standard records don't support protected content. Of course, the point is not to have their own VMT, but to make the content of records visible only in the module of their declaration and in modules with declarations of records inheriting from them. If that were the case, I would use regular records to achieve sensible encapsulation of simple data structures, and not bother with TP objects.

But as we know, records not only do not support protected content, but also the inheritance itself.
Title: Re: Classic objects — does FPC generate additional code?
Post by: TRon on December 13, 2024, 11:21:12 pm
It's a pity that standard records don't support protected content. Of course, the point is not to have their own VMT, but to make the content of records visible only in the module of their declaration and in modules with declarations of records inheriting from them. If that were the case, I would use regular records to achieve sensible encapsulation of simple data structures, and not bother with TP objects.
huh ? afaik that mechanism is in place with advanced records, e.g. private/public parts. Or is that not what you meant ?
Title: Re: Classic objects — does FPC generate additional code?
Post by: flowCRANE on December 13, 2024, 11:22:26 pm
TRon: you are right but, advanced records does not support protected sections and does not supports inheritance.

These are the two things I would like to have available for regular records. Currently, I hide the record fields by declaring them in private sections (one section for all the record content), and I achieve inheritance, protected content, and binary compatibility with records by using TP objects.
Title: Re: Classic objects — does FPC generate additional code?
Post by: TRon on December 13, 2024, 11:28:53 pm
Ah ok, yes. Got it now (I should have read the thread with more attention, sorry about that)

To get back to your original question: Isn't that visible with looking at the generated assembler code ?

Title: Re: Classic objects — does FPC generate additional code?
Post by: flowCRANE on December 13, 2024, 11:35:02 pm
I've checked the assembly for various examples and found nothing suspicious. In addition, SizeOf always returns the correct size of a TP object, which is always the sum of its fields (plus alignment). So, so far, I haven't found any evidence that a TP object containing only fields is not binary compatible with records.
Title: Re: Classic objects — does FPC generate additional code?
Post by: TRon on December 14, 2024, 12:37:50 am
So, so far, I haven't found any evidence that a TP object containing only fields is not binary compatible with records.
Be very careful making any assumptions.

Indeed better rely on one of the core developers to provide a more clear answer. Even if things look like something it is not guaranteed to be the same in the (near) future).

fwiw: I understand your predicament as I worked on a golang translation which seem to support records with inheritance. I eventually solved that by using a managed record that contained a class (it was the quickest (and dirtiest) solution at the time).
Title: Re: Classic objects — does FPC generate additional code?
Post by: flowCRANE on December 14, 2024, 01:07:41 am
Indeed better rely on one of the core developers to provide a more clear answer.

Exactly.

Quote
Even if things look like something it is not guaranteed to be the same in the (near) future).

I doubt their memory layout will change in the future. It probably hasn't changed since they were introduced in the language. But even if it did, I can still use the previous compiler version (i.e. 3.2.2) for another 10-20 years, as long as it produces correct executables for the given platform.

Quote
I eventually solved that by using a managed record that contained a class (it was the quickest (and dirtiest) solution at the time).

The simplest way to implement inheritance in currently supported records (even simple ones) is to declare a field with the same type as the base type:

Code: Pascal  [Select][+][-]
  1. type
  2.   TParent = record
  3.     // fields
  4.   end;
Code: Pascal  [Select][+][-]
  1. type
  2.   TChild = record
  3.     Parent: TParent;
  4.     // fields
  5.   end;

Considering that in practice these two records would be declared in separate units, there are two problems that I want to avoid:


Honestly, I could use an additional Parent namespace to clearly see in the code which variables come from the parent type and which from the inherited type, but the problem is that each new inheritance adds a new Parent namespace, so with a longer inheritance path, the syntax for accessing the fields of this first parent record would require something like Parent.Parent.Parent.SomeField, which is absurd. But still the lack of possibility to encapsulate these structures is a hindrance for me.

There would be no problem if advanced records supported protected sections but they don't.

So, to sum up, I'll stick with TP objects for now because they do the job. However, I will keep in mind that their memory layout is not guaranteed.
Title: Re: Classic objects — does FPC generate additional code?
Post by: korba812 on December 14, 2024, 01:17:48 am
The documentation states:
Quote
Objects are stored in memory just as ordinary records with an extra field: a pointer to the Virtual Method Table (VMT). This field is stored first, and all fields in the object are stored in the order they are declared (with possible alignment of field addresses, unless the object was declared as being packed).
https://www.freepascal.org/docs-html/prog/progsu165.html

It seems that objects are like records with an eventual additional field for a pointer to the VMT. On the other hand, the documentation about classes also likens classes to records.
https://www.freepascal.org/docs-html/prog/progsu166.html#x210-2240008.2.13
However, none mentions the possible reordering of fields with optimization. Maybe it is in another place.
Title: Re: Classic objects — does FPC generate additional code?
Post by: flowCRANE on December 14, 2024, 01:26:27 am
Thank you @korba812 for this quote — so it turns out that the memory layout for object is defined and guaranteed. :D
Title: Re: [SOLVED] Classic objects — does FPC generate additional code?
Post by: flowCRANE on December 14, 2024, 01:34:01 am
However, none mentions the possible reordering of fields with optimization. Maybe it is in another place.

Reordering is probably the same as for records, so probably the information about ORDERFIELDS optimization will also apply to objects. But that's not a problem, because I use O3 optimization and ORDERFIELDS is in O4. And even if I were to use O4, I can always disable reordering with {$OPTIMIZATION ORDERFIELDS-}.
Title: Re: Classic objects — does FPC generate additional code?
Post by: Warfley on December 14, 2024, 01:36:29 am
As I already pointed out, this text is at least for classes not correct, as field reordering was introduced in FPC 3.0.0

Specifically compare the documentation of objects and classes to the one of records, where half the documentation page is just concerned with memory layout, while for classes and objects it's just one introductory sentence.
So the record documentation is really clear that the layout is very well defined, including alignment, etc. There is nothing comparable in the documentation for classes and objects
Title: Re: [SOLVED] Classic objects — does FPC generate additional code?
Post by: flowCRANE on December 14, 2024, 01:48:30 am
So the record documentation is really clear that the layout is very well defined, including alignment, etc. There is nothing comparable in the documentation for classes and objects

IMO a detailed description of the memory layout for objects and classes does not exist because it does not make sense to write the same thing in many places. Since objects and classes, as the documentation said, are ordinary records (with an additional field for a pointer to the VMT), there is no point in describing it — those interested can simply go to the documentation of records and learn the memory layout there.

However, if you think that the documentation is not up to date, it is a good idea to ask the developers to confirm this.
Title: Re: [SOLVED] Classic objects — does FPC generate additional code?
Post by: TRon on December 14, 2024, 02:19:36 am
Some comments that might be of interest #40986 (https://gitlab.com/freepascal.org/fpc/source/-/issues/40986)
Title: Re: [SOLVED] Classic objects — does FPC generate additional code?
Post by: Warfley on December 14, 2024, 01:46:08 pm
Since objects and classes, as the documentation said, are ordinary records (with an additional field for a pointer to the VMT), there is no point in describing it — those interested can simply go to the documentation of records and learn the memory layout there.

However, if you think that the documentation is not up to date, it is a good idea to ask the developers to confirm this.
But it's not, besides field reordering have classes different alignment rules (emg. They are not influenced by packrecords directive). So I just know from the top of my head two differences between records and classes, I would bet there will be more. I would not take that single sentence from the documentation as normative, it's the introductory sentence to describe the concept of classes, not to be interpreted to literally
Title: Re: [SOLVED] Classic objects — does FPC generate additional code?
Post by: flowCRANE on December 14, 2024, 04:47:01 pm
You look for a problem where there is none.

Just because a class instance is represented by a regular record (plus a pointer on the VMT) doesn't automatically mean that all the rules and optimizations that apply to records apply to classes.

It is explicitly written that memory layout expects the use of alignment unless the object is declared as packed. There is no mention of how object fields are aligned, so we need to check how the compiler behave to be sure. I've checked this — in the case of both objects and classes, the {$PACKRECORDS} directive is applicable and indeed allows you to specify memory alignment of fields (as long as the objects are not packed).

Code: Pascal  [Select][+][-]
  1. {$mode objfpc}{$PackRecords 1}
  2.  
  3. type
  4.   TTestRecord = record
  5.     A: UInt16;
  6.     B: UInt32;
  7.     C: UInt8;
  8.   end;
  9.  
  10. type
  11.   TTestObject = object
  12.     A: UInt16;
  13.     B: UInt32;
  14.     C: UInt8;
  15.   end;
  16.  
  17. type
  18.   TTestClass = class
  19.     A: UInt16;
  20.     B: UInt32;
  21.     C: UInt8;
  22.   end;
  23.  
  24. begin
  25.   WriteLn('Record: ', SizeOf(TTestRecord));
  26.   WriteLn('Object: ', SizeOf(TTestObject));
  27.   WriteLn('Class:  ', TTestClass.InstanceSize);
  28. end.

Output:

Code: Pascal  [Select][+][-]
  1. Record: 7
  2. Object: 7
  3. Class:  15 // +8 bytes for the VMT pointer

And if I change alignment to 4, the result is as follows:

Code: Pascal  [Select][+][-]
  1. Record: 12
  2. Object: 12
  3. Class:  20 // +8 bytes for the VMT pointer

The lack of information about packing these data structures in the documentation is a minor inconvenience, but the compiler works according to what is in the documentation - objects and classes are ordinary records, so you can influence their memory layout with a directive intended for records.

However, the {$Optimization ORDERFIELDS} seems to work only in the case of classes.
Title: Re: Classic objects — does FPC generate additional code?
Post by: PascalDragon on December 14, 2024, 05:05:41 pm
Quote
I'm not sure using getmen and freemem handle fields with automated types. You would have to add initialize and finalize calls for that.

This is not a problem, because I always declare an initialization and finalization function, as well as an allocating and deallocating function for each record and object. This way I can freely use such data structures and allocate them on both the stack and the heap, regardless of whether they contain fields that require initialization/finalization.

Unless your initialization functions also contain calls to FPC's System.Initialize() intrinsic then this is a very dangerous as for example you doing an assignment of an empty string to a AnsiString variable (or field) expects that variable to either be Nil or pointing to a valid AnsiString structure. If it points to garbage (which is what GetMem will allocate) then your code might crash depending on what had been returned by GetMem.

And your finalization functions not using System.Finalize will lead to memory leaks, because FreeMem alone will not clear memory of managed types.
Title: Re: [SOLVED] Classic objects — does FPC generate additional code?
Post by: PascalDragon on December 14, 2024, 05:16:50 pm
The lack of information about packing these data structures in the documentation is a minor inconvenience, but the compiler works according to what is in the documentation - objects and classes are ordinary records, so you can influence their memory layout with a directive intended for records.

The lack of information means that this is an implementation detail. No matter what results you get if there is a need then these implementation details can (and do) change. They are left undocumented on purpose to allow for the compiler to generate better code in the future. That's also why optimizations like ORDERFIELDS exist, because the memory layout of a class instance is up to the compiler. That the alignment directives currently influence the memory layout of classes as well is not set in stone, but just a simple by-product of how structured types (records, objects, classes) are handled inside the compiler.
Title: Re: [SOLVED] Classic objects — does FPC generate additional code?
Post by: flowCRANE on December 14, 2024, 05:25:15 pm
Unless your initialization functions also contain calls to FPC's System.Initialize() intrinsic then this is a very dangerous […]

My initialization function sets a default value for each field of the object. I don't use the String type, or any other type that magically gets initialized (by the additional code generated by the compiler). In fact, the operation of my initialization functions can be simplified to a call to AllocMem (to get memory block for the object filled with zeros), except that sometimes the default value of some field is e.g. -1 instead of 0.

But that's when it comes to allocating objects on the heap, and I don't do that very often. Most of the objects I use are allocated either on the stack (local variables) or in the data segment (global variables). Some of these global variables are just ordinary arrays of objects, where each object needs to be initialized and finalized (using my functions).

Quote
And your finalization functions not using System.Finalize will lead to memory leaks, because FreeMem alone will not clear memory of managed types.

Since I don't use data types that needs to be silently initialized and finalized by the compiler, I don't have to use System.Initialize and System.Finalize. Of course, the HeapTrc module checks for me on the fly whether there are leaks or not — I don't have any.

However, from what I've researched, even if I were to use managed-typed fields, I still wouldn't need to use System.Initialize and System.Finalize, since in the object initialization function I set the default values ​​for all fields myself, and in the finalization function I free the resources that were allocated during the object initialization and/or use.
Title: Re: [SOLVED] Classic objects — does FPC generate additional code?
Post by: PascalDragon on December 16, 2024, 10:40:53 pm
Since I don't use data types that needs to be silently initialized and finalized by the compiler, I don't have to use System.Initialize and System.Finalize.

As long as you don't use managed types just using GetMem and FreeMem will indeed work together with you manually initializing/finalizing the fields.

However, from what I've researched, even if I were to use managed-typed fields, I still wouldn't need to use System.Initialize and System.Finalize, since in the object initialization function I set the default values ​​for all fields myself, and in the finalization function I free the resources that were allocated during the object initialization and/or use.

That's where you're wrong. If you allocate with e.g. GetMem you get memory with essentially random content. By assigning a value to a field of a managed type the previous value of the field will first be finalized and that requires the value of the field to be a valid value which a random value most likely won't be. System.Initialize however will directly initialize the fields without requiring the previous value to be finalized.
TinyPortal © 2005-2018