Recent

Author Topic: Managed Objects  (Read 8382 times)

Warfley

  • Hero Member
  • *****
  • Posts: 1863
Managed Objects
« on: June 13, 2023, 02:26:58 pm »
I really like managed records, because they allow to avoid the cumbersome try-finally-free constructs always present. For example, just reading something from a file requires the following with TFileStream:
Code: Pascal  [Select][+][-]
  1. fs := TFileStream.Create(FileName, fmOpenRead);
  2. try
  3.   str := fs.ReadAnsiString;
  4. finally
  5.   fs.Free;
  6. end;

Of those 6 lines, 4 (i.e. 1/3) are just dedicated to the obligatory closing of the file without giving any additional information to the programmer. Basically this is just boilerplate code that makes the actualy functionality less readable.

You can use a managed record instead, which uses the Finalize Operator to close the file:
Code: Pascal  [Select][+][-]
  1. type
  2.   TFileStreamRec = record
  3.     FFile: THandle;
  4.     class operator Finalize(var rec: TFileStreamRec);  // Closes the file
  5.     function Open(FileName: String);  // Opens the file
  6.     function ReadString: String;
  7.   end;
  8.  
  9. [...]
  10.  
  11. // Usage
  12. fs.Open(FileName);
  13. str := fs.ReadString;

This is much easier to use and results in much smaller code with much less clutter (Just 2 meaningful lines, instead of 6 with 4 meaningless lines).

But the problem here is that Records do not support inheritance, and the power of TFileStream is exactly that. For example many classes like TStrings have a LoadFromStream method, where you can pass a any kind of stream, this allows to read data from a Network connection (TSocketStream), from memory (TMemoryStream) or from a file (TFileStream).

Oldschool objects which are located on the stack allow for inheritance, but don't allow for management operators.
But it is possible with a hack to also get managemant operators from Managed Records to also work in an Object. When an object has a managed record as member, then the compiler will call the records management operators when the objects gets initialized, finalized or copied.
Simultaniously, the first member of an object will be at offset 0 of the objects memory, meaning that the pointer of that field is equivalent to the pointer of the object. This allows to simply call functions on an object from that record.

With this I've built a small proof of concept. It tries to emulate the TStream inheritance hierachy to show that this actually works as with TFileStream shown above:
Code: Pascal  [Select][+][-]
  1. program project1;
  2.  
  3. {$mode objfpc}{$H+}
  4. {$ModeSwitch advancedrecords}
  5.  
  6. uses
  7.   SysUtils;
  8.  
  9. type
  10.   generic TAutoDone<T> = record
  11.   class operator Finalize(var rec: specialize TAutoDone<T>);
  12.   end;
  13.  
  14. class operator TAutoDone.Finalize(var rec: specialize TAutoDone<T>);
  15. type
  16.   PT = ^T;
  17. begin
  18.   PT(@Rec)^.Done;
  19. end;
  20.  
  21. type
  22.   TStream = object      
  23.   private
  24.     AutoDone: specialize TAutoDone<TStream>;
  25.   public
  26.     function Read(var Buff; Size: SizeInt): SizeInt; virtual; abstract;
  27.     function Write(const Buff; Size: SizeInt): SizeInt; virtual; abstract;
  28.  
  29.     function ReadLn: String;
  30.     procedure WriteLn(s: String);
  31.  
  32.     destructor Done; virtual;
  33.   end;
  34.  
  35. function TStream.ReadLn: String;
  36. var
  37.   c: Char;
  38. begin
  39.   Result := '';
  40.   repeat
  41.     if Read(c, 1) <> 1 then
  42.       Exit;
  43.     Result += c;
  44.   until c = #10;
  45.   Result := Result.Trim;
  46. end;
  47.  
  48. procedure TStream.WriteLn(s: String);
  49. begin
  50.   s += LineEnding;
  51.   Write(s[1], s.Length);
  52. end;
  53.  
  54. destructor TStream.Done;
  55. begin
  56.   // To be overriden
  57. end;
  58.  
  59. type
  60.   TIOStream = object(TStream)
  61.   private
  62.     FInFile: THandle;
  63.     FOutFile: THandle;
  64.   public
  65.     function Read(var Buff; Size: SizeInt): SizeInt; virtual;
  66.     function Write(const Buff; Size: SizeInt): SizeInt; virtual;
  67.  
  68.     constructor Init(InHandle, OutHandle: THandle);
  69.   end;
  70.  
  71. function TIOStream.Read(var Buff; Size: SizeInt): SizeInt;
  72. begin
  73.   Result := FileRead(FInFile, Buff, Size);
  74. end;
  75.  
  76. function TIOStream.Write(const Buff; Size: SizeInt): SizeInt;
  77. begin
  78.   Result := FileWrite(FOutFile, Buff, Size);
  79. end;
  80.  
  81. constructor TIOStream.Init(InHandle, OutHandle: THandle);
  82. begin
  83.   FInFile := InHandle;
  84.   FOutFile := OutHandle;
  85. end;
  86.  
  87. type
  88.   TConsoleStream = object(TIOStream)
  89.   public
  90.     constructor Init;
  91.   end;
  92.  
  93. constructor TConsoleStream.Init;
  94. begin
  95.   inherited Init(StdInputHandle, StdOutputHandle);
  96. end;
  97.  
  98. type
  99.   TFileStream = object(TIOStream)
  100.   private
  101.     FFile: THandle;
  102.   public
  103.     constructor Open(const FileName: String);
  104.  
  105.     destructor Done; virtual;
  106.   end;
  107.  
  108. constructor TFileStream.Open(const FileName: String);
  109. begin
  110.   if not FileExists(FileName) then
  111.     FFile := FileCreate(FileName)
  112.   else
  113.     FFile := FileOpen(FileName, fmOpenReadWrite);
  114.   FileSeek(FFile, 0, 0);
  115.   inherited Init(FFile, FFile);
  116. end;
  117.  
  118. destructor TFileStream.Done;
  119. begin
  120.   FileClose(FFile);
  121. end;
  122.  
  123. procedure TestConsoleIO;
  124. var
  125.   stream: TConsoleStream;
  126.   s: String;
  127. begin
  128.   stream.Init;
  129.   s := Stream.ReadLn;
  130.   stream.WriteLn('Hello: ' + s);
  131. end;
  132.  
  133. procedure TestFileWrite;
  134. var
  135.   fs: TFileStream;
  136. begin
  137.   fs.Open('test.txt');
  138.   fs.WriteLn('Test Line');
  139. end;
  140.  
  141. procedure TestFileRead;
  142. var
  143.   fs: TFileStream;
  144. begin
  145.   fs.Open('test.txt');
  146.   WriteLn(fs.ReadLn);
  147. end;
  148.  
  149. begin
  150.   TestConsoleIO;
  151.   TestFileRead;
  152.   TestFileWrite;
  153. end.

There are a few limitations. First the AutoDone must always be the first field, so it must be defined in the base class. But this can be handled by having a virtual destructor. This trick does not work for an analogous AutoInit, because the VMT is constructed by the constructor and so no virtual methods can be used.

Note that TAutoDone has no members, therefore it's size in the TStream is 0. This means that other records that use the same pointer casting trick to get their parents pointer, can be appended afterwards (e.g. to handle the Copy or AddRef operators).

This is a stupid hack and probably not intended behavior. But I just wanted to put this out there because I find this interesting, because I defenetly believe that:
Code: Pascal  [Select][+][-]
  1.   fs.Open('test.txt');
  2.   WriteLn(fs.ReadLn);
  3.  
Is much more readable than:
Code: Pascal  [Select][+][-]
  1.   fs := TFileStream.Create
  2.   try
  3.     WriteLn(fs.ReadLn);
  4.   finally
  5.     fs.Free;
  6.   end;

Thaddy

  • Hero Member
  • *****
  • Posts: 16520
  • Kallstadt seems a good place to evict Trump to.
Re: Managed Objects
« Reply #1 on: June 13, 2023, 04:47:19 pm »
Is a wheel an oxacon, a hexacon or any other approximation of a circle?
That looks a hellofalot like the code me and avk - two implementations of the same thing, independent- wrote some years ago...
On this forum, smart pointers...
But I am sure they don't want the Trumps back...

Warfley

  • Hero Member
  • *****
  • Posts: 1863
Re: Managed Objects
« Reply #2 on: June 13, 2023, 05:39:06 pm »
It's different to a smart pointer in the sense that it comes from inside the type, while a smartpointer wraps a type from the outside.

Basically you can use any type and wrap it in a smart pointer, but this requires additional code at the callsite, while this way you design your type to be automatically finalized, but means you can't use the type without this. It streamlines the usage to be fully transparent, but requires the types to be specifically built for this.

Take a generic smartpointer implementation:
Code: Pascal  [Select][+][-]
  1. type
  2.   TRefCountedFileStream = specialize TRefCountedClass<TFileStream>;
  3.  
  4. var
  5.   fs: TRefCountedFileStream;
  6. begin
  7.   fs := TFileStream.Create(FileName, fmOpenRead);
  8.   str := fs.get.ReadAnsiString;
  9. end;
There are a few inconviniences, first TRefCountedFileStream must be specifically specialized (you can use the specialization inline but this makes the code less readable). Then secondly there is a bit of a break in that the type that fs is, is different from the type where you call create. This is not big but can be annoying (for example when I create a bunch of class fields in a constructor, I just copy the definitions, change : to := and add a .Create behind them). Lastly, whenever you use the reference counted object you need to somehow dereference it, e.g. via a "get" method.
Note that you can also always get the object pointer through .get and use it without reference counting, which can subvert the reference counting (basically alows use after free).

Again the big advantage is that it works with any class out of the box, but it adds another layer on the callsite to do so.

The other big difference is that smart pointers do not map on inheritance structures. Take the following example with the code above:
Code: Pascal  [Select][+][-]
  1. function ReadToEnd(constref AStream: TStream): String;
  2. var
  3.   buff: String = '';
  4.   Len: SizeInt;
  5. begin
  6.   Result := '';
  7.   repeat
  8.     SetLength(buff, 1024);
  9.     Len := AStream.Read(buff[1], 1024);
  10.     SetLength(buff, Len);
  11.     Result += Buff;
  12.   until buff.Length < 1024;
  13. end;
  14.  
  15. procedure TestFileRead;
  16. var
  17.   fs: TFileStream;
  18. begin
  19.   fs.Open('test.txt');
  20.   WriteLn(ReadToEnd(fs));
  21. end;

Doing the same with smart pointers fails:
Code: Pascal  [Select][+][-]
  1. function ReadToEnd(AStream: specialize TRefCountedClass<TStream>): String;
  2. var
  3.   buff: String = '';
  4.   Len: SizeInt;
  5. begin
  6.   Result := '';
  7.   repeat
  8.     SetLength(buff, 1024);
  9.     Len := AStream.get.Read(buff[1], 1024);
  10.     SetLength(buff, Len);
  11.     Result += Buff;
  12.   until buff.Length < 1024;
  13. end;
  14.  
  15. procedure TestFileRead;
  16. var
  17.   fs: specialize TRefCountedClass<TFileStream>;
  18. begin
  19.   fs := TFileStream.Create('test.txt', fmOpenRead);
  20.   WriteLn(ReadToEnd(fs)); // error because TRefCountedClass<TFileStream> does not inherit from TRefCountedClass<TStream>
  21. end;
Meaning to use inheritance, you must always strip away the reference counting first and pass the raw pointer

Bad Sector

  • Jr. Member
  • **
  • Posts: 69
    • Runtime Terror
Re: Managed Objects
« Reply #3 on: June 13, 2023, 07:50:53 pm »
Yeah, having objects be able to do pretty much everything classes and records are able to do (and the opposite, where it makes sense) is one of my biggest wishes with the language.

Currently Free Pascal has three compound types that all can do some of the stuff the other compound types can do but not all of the stuff: you want inheritance? Class and object will work, but not record. You want an instance to be allocated as part of an another type instead of on the heap? Object and record will work, but not class. You want to have the compiler to handle lifetime management? Record will work but not class or object. Want RTTI reflection? Class will work but not object or record. I also remember having the compiler complain when an operator was declared as "class operator" (i.e. explicitly passing both parameters) vs "operator" (i.e. implicitly passing the first parameter) when changing a type from "object" to "record" or vice versa - why does it even matter?

These are all highly inconsistent.

I used to think this is because the compiler implements these separately so the FPC developers will need to copy/paste a bunch of stuff, but when i looked in the code some time ago i noticed that all compound objects use largely the same code. So i don't know why such unnecessary and pointless restrictions are in place.

Like you i use a record with managed operators to get management functionality in objects but this is a workaround that makes the code unnecessarily complex (especially when you want a generic object with management operators), verbose and shouldn't be needed in the first place.
Kostas "Bad Sector" Michalopoulos
Runtime Terror

Warfley

  • Hero Member
  • *****
  • Posts: 1863
Re: Managed Objects
« Reply #4 on: June 13, 2023, 08:04:55 pm »
My theory is simply the following:
1. Objects are outdated and only for backwards compatibility and won't be touched and don't receive any new features.
2. Classes are what everything is built on so their inherit limitations (e.g. that they are always just pointers in disguise and mostly only useful on the heap and require manual memory management) cannot be simply overcome
Therefore 3. records is where all the new and fancy features are going to be explored and implemented.

So while records get all the new and shiny stuff like (generic) operator overloading, management operators and soon also RTTI, their lack of inheritance can make them feel extremely limited in their usability.

Thaddy

  • Hero Member
  • *****
  • Posts: 16520
  • Kallstadt seems a good place to evict Trump to.
Re: Managed Objects
« Reply #5 on: June 13, 2023, 08:50:35 pm »
It's different to a smart pointer in the sense that it comes from inside the type, while a smartpointer wraps a type from the outside.
No, both our latest solutions operate from the inside. Obviously.
But I am sure they don't want the Trumps back...

Warfley

  • Hero Member
  • *****
  • Posts: 1863
Re: Managed Objects
« Reply #6 on: June 13, 2023, 08:59:11 pm »
Which version do you mean then?

jamie

  • Hero Member
  • *****
  • Posts: 6800
Re: Managed Objects
« Reply #7 on: June 14, 2023, 01:54:37 am »
IInterfaces ?

It at least will call the Destructor for you.

I did ask for a option to have a REFERENCE to return in a function where as return would be a pointer to the actual returning holder of the object being created.

 With this, calling any constructor as a function could be setup to do this thus after returning the instance will be defined without using  the normal  Instance := TClass.Constructor.

 Oh well, looks to me you are leaning towards C++ type classes  :o

« Last Edit: June 14, 2023, 02:00:53 am by jamie »
The only true wisdom is knowing you know nothing

Warfley

  • Hero Member
  • *****
  • Posts: 1863
Re: Managed Objects
« Reply #8 on: June 14, 2023, 07:49:01 am »
Yeah the usage of interfaces is also quite useful for this, and I have used this a few times in the past.
The issue with interfaces is simply that you have to write your type definition twice (at least the public part) once as the interface and once the actual implementating class.

Note that I don't think that this object version is an attempt to provide a bit more in the direction of managed records, but with inheritance. I don't think this is necessarily useful in many cases, in most situations it may be more practical to use an interface. But I think that this is still an interesting idea, which is why I shared it

Bad Sector

  • Jr. Member
  • **
  • Posts: 69
    • Runtime Terror
Re: Managed Objects
« Reply #9 on: June 14, 2023, 03:32:42 pm »
My theory is simply the following:
1. Objects are outdated and only for backwards compatibility and won't be touched and don't receive any new features.
2. Classes are what everything is built on so their inherit limitations (e.g. that they are always just pointers in disguise and mostly only useful on the heap and require manual memory management) cannot be simply overcome
Therefore 3. records is where all the new and fancy features are going to be explored and implemented.

So while records get all the new and shiny stuff like (generic) operator overloading, management operators and soon also RTTI, their lack of inheritance can make them feel extremely limited in their usability.

IMO the whole idea of a core language feature like "object" being obsolete makes no sense - especially when taking into account FPC's modeswitch feature. Besides, objects have a lot of features that didn't exist in Turbo Pascal - you can even make generic objects, in fact this is how my own containers work to avoid creating stuff in the heap.

While i can see classes being hard to refactor due to backwards compatibility, since there is all that new stuff added to records, then at least records and objects should become the exact same thing as there are very few things that differentiate between the two anyway (lack of inheritance in records is the main one).
Kostas "Bad Sector" Michalopoulos
Runtime Terror

PascalDragon

  • Hero Member
  • *****
  • Posts: 5855
  • Compiler Developer
Re: Managed Objects
« Reply #10 on: June 14, 2023, 11:23:13 pm »
IMO the whole idea of a core language feature like "object" being obsolete makes no sense - especially when taking into account FPC's modeswitch feature. Besides, objects have a lot of features that didn't exist in Turbo Pascal - you can even make generic objects, in fact this is how my own containers work to avoid creating stuff in the heap.

Regarding features specific to object we indeed consider them legacy types and are not interested in extending them. They often get features on the side however due to the fact that - as you discovered - quite a lot of code in the compiler is shared between TP-style objects and Delphi-style classes and not explicitly guarding against usage with object.

While i can see classes being hard to refactor due to backwards compatibility, since there is all that new stuff added to records, then at least records and objects should become the exact same thing as there are very few things that differentiate between the two anyway (lack of inheritance in records is the main one).

No, because TP-style objects still have more in common with Delphi-style classes than with (advanced) records. There is no interest in changing this.

SymbolicFrank

  • Hero Member
  • *****
  • Posts: 1315
Re: Managed Objects
« Reply #11 on: June 15, 2023, 09:17:55 am »
Yes, inheriting from TInterfacedObject is simplest, but it's a lot harder for some external library to determine if the object/class is still holding an active file handle.

Bad Sector

  • Jr. Member
  • **
  • Posts: 69
    • Runtime Terror
Re: Managed Objects
« Reply #12 on: July 01, 2023, 10:25:16 pm »
No, because TP-style objects still have more in common with Delphi-style classes than with (advanced) records. There is no interest in changing this.

The thing is that as i wrote previously, objects are currently the only way to have both inheritance and are value types - i.e. unlike classes you can allocate them in the stack and have them as part of another compound type (be it another object or class). This helps a lot in cases where memory usage (both in terms of resource usage and how the memory is accessed) and lifetime management are concerned (e.g. unlike a class, you can't accidentally have a Nil object). At the same time management operators are helpful - especially in cases like, e.g, custom containers and handles for references, but those are only available in records.

It is frustrating to not have one type that can do everything what both objects and advanced records can do and the distinction between the two feels completely arbitrary.
Kostas "Bad Sector" Michalopoulos
Runtime Terror

Thaddy

  • Hero Member
  • *****
  • Posts: 16520
  • Kallstadt seems a good place to evict Trump to.
Re: Managed Objects
« Reply #13 on: July 02, 2023, 07:28:21 am »
You can actually allocate classes on the stack as I explained before:
https://forum.lazarus.freepascal.org/index.php/topic,42282.msg294908.html#msg294908

There are certain limitations.
« Last Edit: July 02, 2023, 07:30:46 am by Thaddy »
But I am sure they don't want the Trumps back...

Ryan J

  • Full Member
  • ***
  • Posts: 138
Re: Managed Objects
« Reply #14 on: July 06, 2023, 03:45:23 pm »
No, because TP-style objects still have more in common with Delphi-style classes than with (advanced) records. There is no interest in changing this.

It is frustrating to not have one type that can do everything what both objects and advanced records can do and the distinction between the two feels completely arbitrary.

Indeed it's a glaring problem with the design. Things we all agree on:

Are records good? yes.
Is inheritance good? Yes.

So then surely it must follow: are records with inheritance good? and the answer is NO, this is not supported and not planned, find another way to do it. It makes no sense and is frustrating to get stuck between the types all of which are incomplete to some degree.

 

TinyPortal © 2005-2018