Print Page - Smart pointers revisited. Let me know what you think.

Programming => General => Topic started by: Thaddy on August 06, 2019, 02:59:07 pm

Title: Smart pointers revisited. Let me know what you think.
Post by: Thaddy on August 06, 2019, 02:59:07 pm

I revisited my old smart pointer code - from this forum - since we have no default parameter for types other than arrays(yet).
I think I came up with a rather nice - my opinion - rewrite. Note I have also a version based on management operators, this is just the interfaced type.
It works only with classes with parameter-less constructors at the moment.

What's new:
- You don't have to use typecasts.
- Syntax for boxed is immediately obvious, more readable, therefor less prone to mix ups
- Hard casts still work too
- Assignments to a T(whateverclass) gives you a reference to the boxed instance
- You can Unbox! before lazy scope ends.

Here's an example with explanation:

Code: Pascal [Select][+]

program smartptrdemo;
{$mode delphi}{$ifdef mswindows}{$apptype console}{$endif}{$H+}
uses classes, smartptrs;
 
type
  TAutoStringlist = Auto<Tstringlist>; 
  
procedure TestLocal;
var
  l:TAutoStringList;  
begin
  l[box].Add('Test me, local');
  writeln(L[box].text);
end;  
 
var
  a,c:TAutoStringList;
  b:Tstringlist;
begin
  // Auto-creates a boxed stringlist if it does not exist yet.
  // You can subsequently refer to the stringlist as a[box]
  // This has the advantage that the syntax makes it clear 
  // that it is a boxed variable, automatically released.
  a[box].add('test');
  writeln('> ' ,a[box].text);
  
  // You can also manually release the boxed stringlist if required
  // This can be an advantage when you want to control the release time
  a[unbox]; 
  
  // This auto-creates a new stringlist that is empty
  // because the previous is unboxed i.e. destroyed.
  // The text property is initially empty, of course.
  writeln('> ' ,a[box].text, '< should be just a space between > and <');     
  a[box].Add('test some more');
  writeln('> ' ,a[box].text);
  
  // test for local variables
  TestLocal;
  
  // Implicit: 
  // If you do not want to use the [box] syntax, declare a TStrings
  // and assign the boxed value to it. No need to create.
  // b is now a reference to c[box] and is the boxed stringlist.
  c:=default(TAutoStringlist); // shut up compiler
  b:=c; 
  
  // Explicit: 
  // You can hardcast Auto<T> to T
  // (Again an alternative syntax)
  TStringlist(c).Add('Test more');
  
  // c is updated, so is b, b is a reference.
  writeln(b.Text);
  // all cleanup is automatic 
end.

And here's the (upgraded, new) unit:

Code: Pascal [Select][+]

unit smartptrs;
{$ifdef fpc}{$mode delphi}{$endif}
interface
{$ifdef fpc}{$push}{$endif}{$interfaces com}
type
  TBoxtype = (Box,UnBox);
 
  Auto<T:class, constructor> = record
  strict private
    FValue:T;
    FFreeTheValue:IInterface;
    function GetValue:T;
    function GetValueFrom(v:TBoxtype):T;
    type
      TFreeTheValue = class(TInterfacedObject)
      private
        fObjectToFree:TObject;
      public
        constructor Create(anObjectToFree: T);
        destructor Destroy;override;
      end;
   public
     constructor Create(AValue: T);overload;
     procedure Create;overload;
     class operator Implicit(var smart: Auto<T>):T;
     class operator Explicit(var smart: Auto<T>):T;
     property value: T read GetValue;
     property ValueFrom[a:TBoxType]:T read GetValueFrom;default;
   end;
{$ifdef fpc}{$pop 'delphi is on its own here'}{$endif}
 
implementation
   
   constructor Auto<T>.TFreeTheValue.Create(anObjectToFree:T);
   begin
     self.fObjectToFree := anObjectToFree;
   end;
   
   destructor Auto<T>.TFreeTheValue.Destroy;
   begin
     fObjectToFree.Free;
     inherited;
   end;
   
   constructor Auto<T>.Create(AValue:T);
   begin
     FValue := AValue;
     FFreeTheValue := TFreeTheValue.Create(FValue);
   end;
   
   procedure Auto<T>.Create;
   begin
     Auto<T>.Create(T.Create);
   end;
      
   class operator Auto<T>.Implicit(var smart: Auto<T>):T;
   begin
     Result := Smart.Value;
   end;
 
   class operator Auto<T>.Explicit(var smart: Auto<T>):T;
   begin
     Result := Smart.Value;
   end;
 
   function Auto<T>.GetValueFrom(v:TBoxType):T;
   begin
     if v = Box then Result:=GetValue else begin FFreeTheValue := nil; end;
   end;
   
   function Auto<T>.GetValue:T;
   begin
     if not Assigned(FFreeTheValue) then
       Self := Auto<T>.Create(T.Create);
     Result := FValue;
   end;
end.

That's very simple code isn't it? :)

P.S: This version is Delphi compatible, the version with management operators is not. (But has basically the same interface section and usage pattern)

Title: Re: Smart pointers revisited. Let me know what you think.
Post by: avra on August 06, 2019, 11:38:45 pm

Nice code. However that boxed/unboxed code looks a little odd. Not that I couldn't adapt, but I do not find it that intuitive and natural as I would like it to be. Maybe it's just me and I do not see it properly, but I find this smart pointers example code more self explaining:
https://adugmembers.wordpress.com/2011/12/05/smart-pointers/

I am not an expert so this remark is not based on a topic knowledge, but more on a personal feeling when looking at smart pointers example code.

Title: Re: Smart pointers revisited. Let me know what you think.
Post by: lainz on August 07, 2019, 12:29:17 am

I prefer thaddy one, since it creates the object automatically.

But a non verbose option is somewhat we want too ;) Because already defined in the type is the behaviour it will have "TAutoStringList" it says it all for me.

I can imagine an FPC automatic memory like this everywhere, it has bug consequences you can think of it?

Title: Re: Smart pointers revisited. Let me know what you think.
Post by: avk on August 07, 2019, 07:14:41 am

Nice, although it looks a bit heavier compared to managed records. It seems that this implementation does not require any additional actions when copying (I believe the implementation with managed records does)? And it seems that the modifier "default" would improve the usefulness of this primitive.

Title: Re: Smart pointers revisited. Let me know what you think.
Post by: Thaddy on August 07, 2019, 07:37:46 am

@avra That implementation is not currently possible (anonymous methods are used). The [box] syntax originated from the current lack of default properties other that array properties.
But it has actually three advantages
- It is immediately obvious you are dealing with a smart class
- It has a very clear [unbox] option, something the code you refer to lacks.
- an easy auto-create, single step. The other code demands a separate step. Both create and destroy are implicit!
@lainz Currently it only works for classes with a parameterless constructor. But indeed, a type specialization is enough.
@avk I will add the alternative implementation with advanced records later today. But that one is not Delphi compatible and that was important to me.
And yes, the default modifier would be great. That's how I came up with the boxunbox syntax (with hindsight, I actually like that...) to work around it.
Most importantly is the single variable instantiation, although assignment is also possible (b:=c in the example)

Thank you for all the remarks.

Title: Re: Smart pointers revisited. Let me know what you think.
Post by: PascalDragon on August 07, 2019, 09:18:21 am

Quote from: avra on August 06, 2019, 11:38:45 pm

Nice code. However that boxed/unboxed code looks a little odd. Not that I couldn't adapt, but I do not find it that intuitive and natural as I would like it to be. Maybe it's just me and I do not see it properly, but I find this smart pointers example code more self explaining:
https://adugmembers.wordpress.com/2011/12/05/smart-pointers/

I am not an expert so this remark is not based on a topic knowledge, but more on a personal feeling when looking at smart pointers example code.

Oh, wow, that is a cheeky way to do this. O.o
I should probably forward that to Blaise to make sure his implementation of anonymous functions supports that as well. :-[

Title: Re: Smart pointers revisited. Let me know what you think.
Post by: Thaddy on August 07, 2019, 09:32:01 am

Quote from: PascalDragon on August 07, 2019, 09:18:21 am

Quote from: avra on August 06, 2019, 11:38:45 pm
Nice code. However that boxed/unboxed code looks a little odd. Not that I couldn't adapt, but I do not find it that intuitive and natural as I would like it to be. Maybe it's just me and I do not see it properly, but I find this smart pointers example code more self explaining:
https://adugmembers.wordpress.com/2011/12/05/smart-pointers/

I am not an expert so this remark is not based on a topic knowledge, but more on a personal feeling when looking at smart pointers example code.
Oh, wow, that is a cheeky way to do this. O.o
I should probably forward that to Blaise to make sure his implementation of anonymous functions supports that as well. :-[

Yes, it is rather nice, furthermore you can build my extensions on top of it, so you still have the single assignment and immediate create. I am experimenting with it.
The one thing I miss out on is just the default for non-array properties. Hence I was more or less forced to use the Variable[box] syntax.

I am also experimenting with using rtti to support more complex constructors. (It will never be lightweight, but as a thought experiment)
Another experiment uses T(TClass) parameters which looks promising.

Title: Re: Smart pointers revisited. Let me know what you think.
Post by: avra on August 07, 2019, 12:05:17 pm

Quote from: Thaddy on August 07, 2019, 07:37:46 am

@avra That implementation is not currently possible (anonymous methods are used).

Bummer.

Quote from: Thaddy on August 07, 2019, 07:37:46 am

The [box] syntax originated from the current lack of default properties other that array properties.

Hopefully we will get other then array default properties so box/unbox can be avoided.

Quote from: Thaddy on August 07, 2019, 07:37:46 am

It has a very clear [unbox] option, something the code you refer to lacks.

I thought this line from Delphi example was unboxing:

Code: Pascal [Select][+]

Person1 := nil; // Release early

If it is, then I find it more natural.

Title: Re: Smart pointers revisited. Let me know what you think.
Post by: Thaddy on August 07, 2019, 01:55:37 pm

Quote from: avra on August 07, 2019, 12:05:17 pm

Quote from: Thaddy on August 07, 2019, 07:37:46 am
@avra That implementation is not currently possible (anonymous methods are used).
Bummer.

Yes. And there is patch from Maciej.

Quote

Hopefully we will get other then array default properties so box/unbox can be avoided.

Yes.

Quote

I thought this line from Delphi example was unboxing:
Code: Pascal [Select][+][-]
Person1 := nil; // Release early
If it is, then I find it more natural.

Here I disagree, handling smartpointers should be recognizable and obvious. So a little syntactic sugar here may be warranted.
But I only realized that *after* I implemented it.

Anyway, the biggest difference is that my smart pointer is a single stop shop, not requiring a second step, which, to my mind, is how it is supposed to be.

I have currently made two changes:
- References to TObject changed to T, which was an oversight. Anyone can do that to the original sourcecode
- A very early version of RTTI (costly)

Title: Re: Smart pointers revisited. Let me know what you think.
Post by: avra on August 07, 2019, 03:56:18 pm

Quote from: Thaddy on August 07, 2019, 01:55:37 pm

Quote
I thought this line from Delphi example was unboxing:
Code: Pascal [Select][+][-]
Person1 := nil; // Release early
If it is, then I find it more natural.

Here I disagree, handling smartpointers should be recognizable and obvious.

I am probably missing something, but it is obvious to me. With a little macro help we can even write it like this:

Code: Pascal [Select][+]

Person1 := kill; // Kill object

Title: Re: Smart pointers revisited. Let me know what you think.
Post by: Thaddy on August 08, 2019, 06:52:46 am

Yes, one could do that. (I actually do that internally.)
Remains the issue that the Delphi example - as is - needs extra work from the side of the programmer anyway.
But that does not mean that the technique I used can 't be incorporated into that sourcecode.

I forgot to mention that you also have - the old - access through the instance.value property if you really do not like the [box] syntax.

Code: Pascal [Select][+]

{$mode delphi}{$H+}
uses classes, smartptrs;
type
  TAutoStream = Auto<Tstringlist>;
var
  s:TAutoStream;  
begin
  // same as s[box].LoadFromFile
  s.Value.LoadFromFile('smartptrs.pas');
  // same as s[box].text
  writeln(s.value.text);
end.

All the more reason to have value as a default property.

Title: Re: Smart pointers revisited. Let me know what you think.
Post by: avra on August 08, 2019, 08:26:54 am

Quote from: Thaddy on August 08, 2019, 06:52:46 am

I forgot to mention that you also have - the old - access through the instance.value property if you really do not like the [box] syntax.

Yes, I would probably use that until Value is able to become default property.

I forgot to say that I much more like your Auto<> naming

Code: Pascal [Select][+]

type
  TAutoPerson = Auto<TPerson>;
var
  Person1: TAutoPerson;

over the one from Delphi example:

Code: Pascal [Select][+]

var
  Person1: ISmartPointer<TPerson>;

Using Auto is natural, clear and self explaining even for someone who hasn't heard of smart pointers before. Using ISmartPointer does not have that benefit. I would even prefer Auto over Managed, which is the only one I can think of with a better self explanation.

Title: Re: Smart pointers revisited. Let me know what you think.
Post by: ASerge on August 08, 2019, 05:26:18 pm

Thaddy, I propose a version where the structure only takes the size of a pointer. Instead of the default property and additional "Value" field, I suggest only one "Self" field.
Unit:

Code: Pascal [Select][+]

unit smartptrs;
 
{$IFDEF FPC}{$MODE DELPHI}{$ENDIF}
 
interface
 
type
  TAuto<T:class, constructor> = record
  strict private
    FHolder: IInterface;
    function GetSelf: T;
    procedure SetSelf(const AValue: T);
    type
      THolder = class(TInterfacedObject)
      public
        Field: T;
        constructor Create(AField: T);
        destructor Destroy; override;
      end;
  public
    constructor Create(AInstance: T);
    class operator Implicit(const Other: TAuto<T>): T;
    property Self: T read GetSelf write SetSelf;
  end;
 
implementation
 
constructor TAuto<T>.THolder.Create(AField: T);
begin
  Field := AField;
end;
 
destructor TAuto<T>.THolder.Destroy;
begin
  Field.Free;
  inherited;
end;
 
constructor TAuto<T>.Create(AInstance: T);
begin
  SetSelf(AInstance);
end;
 
class operator TAuto<T>.Implicit(const Other: TAuto<T>): T;
begin
  Result := Other.Self;
end;
 
function TAuto<T>.GetSelf: T;
begin
  if not Assigned(FHolder) then
    SetSelf(T.Create);
  Result := (FHolder as THolder).Field;
end;
 
procedure TAuto<T>.SetSelf(const AValue: T);
begin
  if Assigned(AValue) then
    FHolder := THolder.Create(AValue)
  else
    FHolder := nil;
end;
 
end.

Example:

Code: Pascal [Select][+]

{$APPTYPE CONSOLE}
{$IFDEF FPC}{$MODE DELPHI}{$ENDIF}
 
uses Classes, smartptrs;
 
type
  TAutoStringList = TAuto<TStringList>;
 
procedure TestLocal;
var
  L: TAutoStringList;
begin
  L.Self.Add('Test me, local');
  Writeln(L.Self.Text);
end;
 
var
  a,c: TAutoStringList;
  b: TStringList;
begin
  // Auto-creates a boxed stringlist if it does not exist yet.
  // You can subsequently refer to the stringlist as a[box]
  // This has the advantage that the syntax makes it clear
  // that it is a boxed variable, automatically released.
  a.Self.add('test');
  Writeln('> ', a.Self.Text);
 
  // You can also manually release the boxed stringlist if required
  // This can be an advantage when you want to control the release time
  a.Self := nil;
 
  // This auto-creates a new stringlist that is empty
  // because the previous is unboxed i.e. destroyed.
  // The text property is initially empty, of course.
  Writeln('> ', a.Self.Text, '< should be just a space between > and <');
  a.Self.Add('test some more');
  Writeln('> ', a.Self.text);
 
  // test for local variables
  TestLocal;
 
  // Implicit:
  // If you do not want to use the [box] syntax, declare a TStrings
  // and assign the boxed value to it. No need to create.
  // b is now a reference to c[box] and is the boxed stringlist.
  c := Default(TAutoStringlist); // shut up compiler
  b := c;
 
  // Explicit:
  // You can hardcast Auto<T> to T
  // (Again an alternative syntax)
  TStringlist(c).Add('Test more');
 
  // c is updated, so is b, b is a reference.
  Writeln(b.Text);
  Readln;
  // all cleanup is automatic
end.

Title: Re: Smart pointers revisited. Let me know what you think.
Post by: Zoran on August 08, 2019, 07:48:28 pm

Aren't these "smart pointers" just C++'s workaround for not having try-finally construct?

Title: Re: Smart pointers revisited. Let me know what you think.
Post by: PascalDragon on August 09, 2019, 09:38:30 am

Quote from: Zoran on August 08, 2019, 07:48:28 pm

Aren't these "smart pointers" just C++'s workaround for not having try-finally construct?

They also allow for automatic memory management. There's simply no need then to manually free object instances except in specific cases (e.g. circular references) and even then that can be simply done by setting the smart pointer variable to Nil.
And it does avoid the clutter that is otherwise introduced by try … finally blocks.

Title: Re: Smart pointers revisited. Let me know what you think.
Post by: Thaddy on August 09, 2019, 11:37:59 am

Quote from: ASerge on August 08, 2019, 05:26:18 pm

Thaddy, I propose a version where the structure only takes the size of a pointer. Instead of the default property and additional "Value" field, I suggest only one "Self" field.

That is possible, but relies on the record layout. First field being the same as the pointer to record itself. IMHO that is implementation detail.

Frankly I can't improve much without compiler support. ( and keep clean code)

I will consider the suggestion, though. Thx.

Title: Re: Smart pointers revisited. Let me know what you think.
Post by: Thaddy on August 10, 2019, 11:51:28 am

I have edited the unit to use T instead of TObject, as previously remarked. (Was an oversight)
Still playing with ASerge's idea.

Title: Re: Smart pointers revisited. Let me know what you think.
Post by: zamronypj on August 10, 2019, 12:57:03 pm

@Thaddy

I am trying to grasp the idea.

Code: Pascal [Select][+]

TAutoStringList = TAuto<TStringList>;

CMIIW, code above works because constructor TStringList.create() does not have any parameters. If constructor has parameters, then how do you pass that required parameters to constructor?

Title: Re: Smart pointers revisited. Let me know what you think.
Post by: Thaddy on August 10, 2019, 01:38:26 pm

As it stands (and I documented in the introduction) it is only for classes with parameter-less constructors.
However, I am working on an RTTI based version. That isn't lightweight, though. This one is.

Also note this is NOT a stop-gap for all memory management, never has been the intention.

Title: Re: Smart pointers revisited. Let me know what you think.
Post by: ASerge on August 10, 2019, 06:53:31 pm

Quote from: zamronypj on August 10, 2019, 12:57:03 pm

If constructor has parameters, then how do you pass that required parameters to constructor?

Code: Pascal [Select][+]

Auto<T:class, constructor> = record

means: T is a class that defines a default constructor (a public parameterless constructor)

Title: Re: Smart pointers revisited. Let me know what you think.
Post by: avk on August 11, 2019, 07:33:30 am

@Thaddy, I didn’t wait for your implementation with managed records and therefore decided to make my own:

Code: Pascal [Select][+]

unit shref;
 
{$mode delphi}
 
interface
 
uses
  SysUtils;
 
type
 
  TGSharedRef<T: class, constructor> = record
  private
    FInstance: T;
    FRefCount: PInteger;
    procedure InitInstance(aValue: T);
    procedure Release;
    function  GetInstance: T;
    procedure SetInstance(aValue: T);
    class operator Initialize(var s: TGSharedRef<T>); inline;
    class operator Finalize(var s: TGSharedRef<T>);
    class operator Copy(constref aSrc: TGSharedRef<T>; var aDst: TGSharedRef<T>); inline;
  public
  type
    TInstance = T;
    class operator Implicit(var s: TGSharedRef<T>): T; inline;
    class operator Explicit(var s: TGSharedRef<T>): T; inline;
    property Instance: T read GetInstance write SetInstance;
  end;
 
implementation
 
procedure TGSharedRef<T>.InitInstance(aValue: T);
begin
  FInstance := aValue;
  if aValue <> nil then
    begin
      New(FRefCount);
      FRefCount^ := 1;
    end;
end;
 
procedure TGSharedRef<T>.Release;
begin
  if FRefCount <> nil then
    begin
      if InterlockedDecrement(FRefCount^) = 0 then
        begin
          Dispose(FRefCount);
          FInstance.Free;
        end;
      FRefCount := nil;
    end;
end;
 
function TGSharedRef<T>.GetInstance: T;
begin
  if FRefCount = nil then
    InitInstance(T.Create);
  Result := FInstance;
end;
 
procedure TGSharedRef<T>.SetInstance(aValue: T);
begin
  if aValue <> FInstance then
    begin
      Release;
      InitInstance(aValue);
    end;
end;
 
class operator TGSharedRef<T>.Initialize(var s: TGSharedRef<T>);
begin
  s.FRefCount := nil;
end;
 
class operator TGSharedRef<T>.Finalize(var s: TGSharedRef<T>);
begin
  s.Release;
end;
 
class operator TGSharedRef<T>.Copy(constref aSrc: TGSharedRef<T>; var aDst: TGSharedRef<T>);
begin
  aDst.Release;
  if aSrc.FRefCount <> nil then
    InterLockedIncrement(aSrc.FRefCount^);
  aDst.FInstance := aSrc.Instance;
  aDst.FRefCount := aSrc.FRefCount;
end;
 
class operator TGSharedRef<T>.Implicit(var s: TGSharedRef<T>): T;
begin
  Result := s.Instance;
end;
 
class operator TGSharedRef<T>.Explicit(var s: TGSharedRef<T>): T;
begin
  Result := s.Instance;
end;
 
end.
 

usage:

Code: Pascal [Select][+]

program sharedref_test;
 
{$mode delphi}
 
uses
  SysUtils, shref;
 
type
 
  TTest = class
  private
    FValue: Integer;
  public
    constructor Create; overload;
    constructor Create(aValue: Integer); overload;
    property Value: Integer read FValue write FValue;
  end;
 
constructor TTest.Create;
begin
  FValue := -1;
end;
 
constructor TTest.Create(aValue: Integer);
begin
  FValue := aValue;
end;
 
var
  GlobRef: TGSharedRef<TTest>;
 
procedure Test;
var
  Ref: TGSharedRef<TTest>;
begin
  WriteLn('begin local proc');
  WriteLn('TTest(Ref).Value = ', TTest(Ref).Value);
  TTest(Ref).Value := 101;
  WriteLn('TTest(Ref).Value = ', TTest(Ref).Value);
  GlobRef := Ref;  // TTest(GlobRef).Value = 101
  Ref.Instance := TTest(nil); //early release
  WriteLn('TTest(Ref).Value = ', TTest(Ref).Value);
  Ref.Instance := TTest.Create(0); //parametized constructor
  WriteLn('TTest(Ref).Value = ', TTest(Ref).Value);
  WriteLn('end local proc');
end;
 
begin
  Test;
  WriteLn('TTest(GlobRef).Value = ', TTest(GlobRef).Value);
  ReadLn;
end.
 

It seems it works?

Title: Re: Smart pointers revisited. Let me know what you think.
Post by: Thaddy on August 11, 2019, 08:52:48 am

Yes. Very similar to mine based on Maciej's earlier scetch.in approach.
And just like mine it suffers from ugly hard casting (until we have a default property, that is ).

This is a good solution for FreePascal, can't be used on Delphi, though,
It generattes also slightly less code, it seems.

Nice!

Title: Re: Smart pointers revisited. Let me know what you think.
Post by: avk on August 12, 2019, 12:35:56 pm

Ok, then maybe it makes sense to add it to LGenerics.

Title: Re: Smart pointers revisited. Let me know what you think.
Post by: Thaddy on August 12, 2019, 12:51:43 pm

Fine with me.

One note: You can not really use the parameterized constructors on things like TFilestream: that does not work. Although for your TTest it works.
I think for that we need to wait for more complete RTTI if at all possible.

Title: Re: Smart pointers revisited. Let me know what you think.
Post by: avk on August 12, 2019, 01:02:20 pm

Of course, if the class does not has a parameterless constructor, then this simply will not compile.

Title: Re: Smart pointers revisited. Let me know what you think.
Post by: Thaddy on August 12, 2019, 01:21:24 pm

Quote from: avk on August 12, 2019, 01:02:20 pm

Of course, if the class does not have a parameterless constructor, then this simply will not compile.

Yes, but it has: TObject has..

Title: Re: Smart pointers revisited. Let me know what you think.
Post by: Zoran on August 12, 2019, 05:42:16 pm

Quote from: Thaddy on August 12, 2019, 01:21:24 pm

Quote from: avk on August 12, 2019, 01:02:20 pm
Of course, if the class does not have a parameterless constructor, then this simply will not compile.
Yes, but it has: TObject has..

Or not: https://forum.lazarus.freepascal.org/index.php/topic,45366.msg327886.html#msg327886
;)

Title: Re: Smart pointers revisited. Let me know what you think.
Post by: Thaddy on August 12, 2019, 05:45:05 pm

Yes. But that technique is not applied.

Title: Re: Smart pointers revisited. Let me know what you think.
Post by: ASerge on August 12, 2019, 06:49:43 pm

Quote from: Thaddy on August 12, 2019, 12:51:43 pm

One note: You can not really use the parameterized constructors on things like TFilestream

Option when the default constructor is not called. For diversity - FPC mode, and as and before, only the size of the pointer.

Code: Pascal [Select][+]

unit Unit1;
 
{$MODE OBJFPC}
{$MODESWITCH ADVANCEDRECORDS}
 
interface
 
type
  generic TAuto<T:class> = record
  strict private type
    TRef = record Item: T; RefCount: LongInt; end;
    PRef = ^TRef;
  strict private
    FRef: PRef;
    class operator Copy(constref Src: TAuto; var Dst: TAuto);
    class operator Finalize(var Self: TAuto); inline;
    class operator Initialize(var Self: TAuto); inline;
    function GetInstance: T; inline;
    procedure Release;
    procedure SetInstance(const Value: T);
  public
    class operator Explicit(constref From: TAuto): T; inline;
    property Instance: T read GetInstance write SetInstance;
  end;
 
implementation
 
class operator TAuto.Copy(constref Src: TAuto; var Dst: TAuto);
begin
  Dst.Release;
  if Assigned(Src.FRef) then
    InterlockedIncrement(Src.FRef^.RefCount);
  Dst.FRef := Src.FRef;
end;
 
class operator TAuto.Explicit(constref From: TAuto): T;
begin
  Result := From.Instance;
end;
 
class operator TAuto.Finalize(var Self: TAuto);
begin
  Self.Release;
end;
 
class operator TAuto.Initialize(var Self: TAuto);
begin
  Self.FRef := nil;
end;
 
function TAuto.GetInstance: T;
begin
  Result := nil;
  if Assigned(FRef) then
    Result := FRef^.Item;
end;
 
procedure TAuto.Release;
begin
  if Assigned(FRef) then
  begin
    if InterlockedDecrement(FRef^.RefCount) = 0 then
    begin
      FRef^.Item.Free;
      Dispose(FRef);
    end;
    FRef := nil;
  end;
end;
 
procedure TAuto.SetInstance(const Value: T);
begin
  if Assigned(FRef) then
  begin
    if Value = FRef^.Item then
      Exit;
    Release;
  end;
  if Assigned(Value) then
  begin
    New(FRef);
    FRef^.Item := Value;
    FRef^.RefCount := 1;
  end
end;
 
end.

Code: Pascal [Select][+]

{$MODE OBJFPC}
{$APPTYPE CONSOLE}
 
uses unit1;
 
type
  TTest = class
  private
    FValue: Integer;
  public
    constructor Create(AValue: Integer); overload;
    property Value: Integer read FValue write FValue;
  end;
 
constructor TTest.Create(AValue: Integer);
begin
  FValue := AValue;
end;
 
type
  TAutoTest = specialize TAuto<TTest>;
var
  GlobRef: TAutoTest;
 
procedure Test;
var
  Ref: TAutoTest;
begin
  WriteLn('begin local proc');
  Ref.Instance := TTest.Create(-1);
  WriteLn('TTest(Ref).Value = ', TTest(Ref).Value);
  TTest(Ref).Value := 101;
  WriteLn('TTest(Ref).Value = ', TTest(Ref).Value);
  GlobRef := Ref;  // TTest(GlobRef).Value = 101
  Ref.Instance := nil; //early release
  Ref.Instance := TTest.Create(0);
  WriteLn('TTest(Ref).Value = ', TTest(Ref).Value);
  WriteLn('end local proc');
end;
 
begin
  Test;
  WriteLn('TTest(GlobRef).Value = ', TTest(GlobRef).Value);
  ReadLn;
end.

Title: Re: Smart pointers revisited. Let me know what you think.
Post by: PascalDragon on August 13, 2019, 09:19:12 am

Quote from: Zoran on August 12, 2019, 05:42:16 pm

Quote from: Thaddy on August 12, 2019, 01:21:24 pm
Quote from: avk on August 12, 2019, 01:02:20 pm
Of course, if the class does not have a parameterless constructor, then this simply will not compile.
Yes, but it has: TObject has..

Or not: https://forum.lazarus.freepascal.org/index.php/topic,45366.msg327886.html#msg327886
;)

I have not yet found a Delphi example that would trigger the constructor-constraint of a generic, thus FPC parses, but ignores that constraint (my assumption is that this is from Delphi.NET times).

Title: Re: Smart pointers revisited. Let me know what you think.
Post by: alpine on January 24, 2022, 05:45:52 pm

My apologies for the necroposting!

Recently, I've decided to take advantage of that smart pointers and I found there is an important operator missing, AddRef.
Cosidering the following usage as ByVal parameter:

Code: Pascal [Select][+]

type
  TAutoTest = specialize TAuto<TTest>;
var
  GlobRef: TAutoTest;
 
procedure A(R: TAutoTest);
begin
  R.Instance.Value := -2;
end;
 
begin
  GlobRef.Instance := TTest.Create(-1); 
  A(GlobRef);
  GlobRef.Instance.Value := -3; // SIGSEGV!
end.
 

It needs an AddRef to increase the reference count, like:

Code: Pascal [Select][+]

type 
  generic TAuto<T: class> = record
   // ...
  strict private
   //...
    class operator AddRef(var Self: TAuto); inline;
   //...
  end; 
 
//...
 
class operator TAuto.AddRef(var Self: TAuto);
begin
  if Assigned(Self.FRef) then
    InterlockedIncrement(Self.FRef^.RefCount);
end;  

Title: Re: Smart pointers revisited. Let me know what you think.
Post by: Thaddy on January 24, 2022, 06:18:58 pm

Do you use {$interfaces corba} by any change? I have never seen the need for that in my TAuto<>. Can you confirm you are using ,y version? and COM? Or are you using any other suggestion from the discussion? In that case I can not help you. My version works different from the code you use, that's why I ask. May be you used the version with management operators? That has issues indeed.

Title: Re: Smart pointers revisited. Let me know what you think.
Post by: Warfley on January 24, 2022, 06:32:20 pm

Note that as long as the double finalize bug is not fixed in the compiler, management operators do not work if they rely on Finalize.

Therefore if you want to use smartpointers you need to use COM interfaces for now.

My bug report to the double finalize issue:
https://gitlab.com/freepascal.org/fpc/source/-/issues/37164
I think I saw another bug report but the search in gitlab is terrible. But basically if a function returns a managed type, it will be finalized twice. As this relies on implicit casts, which are just hidden function calls, these are impacted by this. So if you use the implicit cast to create the smart pointer, it will double finalize and result in dangling pointers

Title: Re: Smart pointers revisited. Let me know what you think.
Post by: alpine on January 24, 2022, 07:06:01 pm

Quote from: Thaddy on January 24, 2022, 06:18:58 pm

Do you use {$interfaces corba} by any change? I have never seen the need for that in my TAuto<>. Can you confirm you are using ,y version? and COM? Or are you using any other suggestion from the discussion? In that case I can not help you. My version works different from the code you use, that's why I ask. May be you used the version with management operators? That has issues indeed.

My observations are based on code from this discussion (for TAuto<>, not for TGSharedRef<>). I'm not using {$interfaces corba}.
But because of Warfley post, may be I should double-check with other compiler versions, mine is not recent.

Title: Re: Smart pointers revisited. Let me know what you think.
Post by: Thaddy on January 24, 2022, 07:15:55 pm

As Warfley observed, the original version( by me) based on an inteface does not suffer the issue.
It is also easier to use because of auto-create (see Tstringlist example)

Title: Re: Smart pointers revisited. Let me know what you think.
Post by: alpine on January 25, 2022, 01:55:18 am

Quote from: Thaddy on January 24, 2022, 07:15:55 pm

As Warfley observed, the original version( by me) based on an inteface does not suffer the issue.
It is also easier to use because of auto-create (see Tstringlist example)

Correct, I can see into the compiler sources that the interfaces are handled quite differently.

I'm currently playing with the TAuto<> class from https://forum.lazarus.freepascal.org/index.php/topic,46306.msg330408.html#msg330408
as they have a reference counter which is of my particular interest. My post was about that generics (I believe yours is named Auto<>).

@Warfley
Although not 100% sure, the issue with the double finalization (in https://gitlab.com/freepascal.org/fpc/source/-/issues/37164) is related to the creation of the hidden temp variable for the enumerator. So with the simplified example given by Serge Anvarov (ASerge?) for the same issue - when the function result is assigned to a variable, the issue dissapears, when discarded - we have double finalization.

So far, I don't see how the ASerge TAuto<> can be affected by the double finalization issue. Still investigating.

Title: Re: Smart pointers revisited. Let me know what you think.
Post by: Thaddy on January 25, 2022, 07:17:20 am

You can get at the refcount from the IInterface by a cast to TInterfacedobject which is the implementing class and has a refcount. I will see if I can surface that..
Add this to the interface based Auto<T>:

Code: Pascal [Select][+]

   function Auto<T>.RefCount:integer;
   begin
     Result := 0;
     if Assigned(FFreeTheValue) then
       Result := (FFreeTheValue as TInterfacedObject).RefCount;
   end;

~~Untested!~~ Somewhat tested. Seems to work.
Now your specialization of Auto<T> has a refcount...
Attached full unit + demo

Again, this is ONLY for the initial interface based Auto<T> by me.

Title: Re: Smart pointers revisited. Let me know what you think.
Post by: Thaddy on January 25, 2022, 10:12:49 am

Note, that with hindsight I kept the original [box/unbox] syntax, because that allows for explicit release, see the attached example above. My later changes where merely to satisfy customers here.. :D

Title: Re: Smart pointers revisited. Let me know what you think.
Post by: alpine on January 25, 2022, 10:56:15 am

Quote from: Thaddy on January 25, 2022, 10:12:49 am

Note, that with hindsight I kept the original [box/unbox] syntax, because that allows for explicit release, see the attached example above. My later changes where merely to satisfy customers here.. :D

Thanks for the effort!

My preferred choice were the advanced records and I presumed all managed objects were treated in the same way by the compiler, too bad it is not the case.
In some other PL, thanks to the templates, multiple inheritance and auto objects, I'm using refcounted objects extensively and that makes my life much easier.

BTW, It is not that the same functionality can't be achieved with TCustomVariantType for example, which in turn is also IInterface by itself. But involving COM and/or Variants sounds like unneeded dead weight to me.

Title: Re: Smart pointers revisited. Let me know what you think.
Post by: Thaddy on January 25, 2022, 11:16:49 am

This particular use of COM interfaces is very lightweight. Everything is solved at compile time, specialization is at type level.
The other option with management operators needs to have the double finalize bug fixed and ideally also default properties for other properties than arrays - also on my wishlist -. The burden would be about the same as using interfaces, in my opinion, after examining the generated assembler.

Title: Re: Smart pointers revisited. Let me know what you think.
Post by: Warfley on January 25, 2022, 11:35:38 am

Quote from: y.ivanov on January 25, 2022, 01:55:18 am

@Warfley
Although not 100% sure, the issue with the double finalization (in https://gitlab.com/freepascal.org/fpc/source/-/issues/37164) is related to the creation of the hidden temp variable for the enumerator. So with the simplified example given by Serge Anvarov (ASerge?) for the same issue - when the function result is assigned to a variable, the issue dissapears, when discarded - we have double finalization.

So far, I don't see how the ASerge TAuto<> can be affected by the double finalization issue. Still investigating.

The interesting bit about the example by Serge is that no copy operation takes place. If we use a constructor also the assignment fails:

Code: Pascal [Select][+]

program Project1;
 
{$mode objfpc}{$H+}
{$ModeSwitch advancedrecords}
 
type
 
  { TSomeManagedRecord }
 
  TSomeManagedRecord = record
    id: Integer;
    class operator Initialize(var e: TSomeManagedRecord);
    class operator Finalize(var e: TSomeManagedRecord);
    class operator AddRef(var e: TSomeManagedRecord);
    class operator Copy(constref s: TSomeManagedRecord; var d: TSomeManagedRecord);
 
    constructor Create(A: Integer);
  end;
var
  nextId: Integer;
 
class operator TSomeManagedRecord.Initialize(var e: TSomeManagedRecord);
begin
  WriteLn('Initialize: ', nextId);
  e.id:=nextId;
  Inc(nextId);
end;
 
class operator TSomeManagedRecord.Finalize(var e: TSomeManagedRecord);
begin
  WriteLn('Finalize: ', e.id);
end;
 
class operator TSomeManagedRecord.AddRef(var e: TSomeManagedRecord);
begin
  WriteLn('AddRef: ', e.id);
end;
 
class operator TSomeManagedRecord.Copy(constref s: TSomeManagedRecord;
  var d: TSomeManagedRecord);
begin
  WriteLn('Copy: ', s.id, '->',d.id);
end;
 
constructor TSomeManagedRecord.Create(A: Integer);
begin
 
end;
 
procedure Test;
var
  R: TSomeManagedRecord;
begin
  R := TSomeManagedRecord.Create(42);
end;
 
begin
  Test;
  Readln;
end.

Result:

Code: Pascal [Select][+]

Initialize: 0
Initialize: 1
Finalize: 1
Copy: 1->0
Finalize: 0
Finalize: 1

So I guess it's due to optimizations (probably RTO) which happens here that the copy is ommited, and that this optimization is not happening when the constructor is used (I already found out that constructors are weird with respect to optimization, e.g. they are never inlined).
So I think the reason why Serges example does not fail is simply because it is so simple that the optimiser circumvents the bug.

Title: Re: Smart pointers revisited. Let me know what you think.
Post by: Thaddy on January 25, 2022, 12:10:28 pm

@Warfley, I am a bit confused here:
1. that example does not actually assigns something useful, like a stringlist.
2. the copy operator is empty, of course there is no copy. In my example the same: only the refcount is increased, since it is a reference.
3. the create is empty too

I have no clue if ASerge's example that you cite is viable, since it seems not complete, whereas my example is a complete example that always works on any class with a default, parameterless, constructor.
I suppose - as discussed and I acknowledged - that eventually this may very well be the way to go, but as it stands my interface approach has its own merits. It is even Delphi compatible.

Maybe ASerge will expand on his example? It would be interesting to see how that works in its entirity.
It is always good to see if any of us has made any progress on the subject: smart pointers can be very useful, e.g. in object persistent frameworks.
I have no particular affinity to any of both options, as long as they work.. :D

Title: Re: Smart pointers revisited. Let me know what you think.
Post by: Warfley on January 25, 2022, 12:32:17 pm

I mean serges example to the bug report, not his example in this post, which was also the one referenced in the quote I replied to (It's confusing I know :D). This is just the minimal example to show the double finalize bug

Title: Re: Smart pointers revisited. Let me know what you think.
Post by: alpine on January 25, 2022, 12:34:39 pm

Quote from: Thaddy on January 25, 2022, 12:10:28 pm

*snip*
I have no clue if ASerge's example that you cite is viable, since it seems not complete, whereas my example is a complete example that always works on any class with a default, parameterless, constructor.
I suppose - as discussed and I acknowledged - that eventually this may very well be the way to go, but as it stands my interface approach has its own merits. It is even Delphi compatible.
*snip

It is actually about two different examples given by ASerge - one is here for TAuto<> and another one is for the double finalization bug issue : https://gitlab.com/freepascal.org/fpc/source/uploads/43c46a9afc8394afd23b490a50216aec/project1.lpr
Warfley talks for the latter one in his latest post.

@Warfley
I'm still for the hidden temporary variables,

Code: Pascal [Select][+]

procedure Test1;
var
  R: TSomeManagedRecord;
begin
  R := TSomeManagedRecord.Create(42);
end;
 
procedure Test2;
var
  R, S: TSomeManagedRecord;
begin
  R := S.Create(42);
end;

Test2 doesn't suffer from the issue. Test1 involves a temporary.

In both cases fpc_copy_proc is not omitted, just fpc_finalize was erroneously called before Create in Test1.

Title: Re: Smart pointers revisited. Let me know what you think.
Post by: Thaddy on January 25, 2022, 12:34:53 pm

Ah, OK. I did not grasp that.

Btw, I attached a renewed version of the example code (auto_refcount.pas) attachment, not the unit. Very small change to explain better, merely comments.

Title: Re: Smart pointers revisited. Let me know what you think.
Post by: Warfley on January 25, 2022, 02:50:11 pm

Quote from: y.ivanov on January 25, 2022, 12:34:39 pm

@Warfley
I'm still for the hidden temporary variables,
Code: Pascal [Select][+][-]
procedure Test1;
var
R: TSomeManagedRecord;
begin
R := TSomeManagedRecord.Create(42);
end;

procedure Test2;
var
R, S: TSomeManagedRecord;
begin
R := S.Create(42);
end;
Test2 doesn't suffer from the issue. Test1 involves a temporary.

In both cases fpc_copy_proc is not omitted, just fpc_finalize was erroneously called before Create in Test1.

The thing is, the constructor should be nothing more than a static class function where instead of Result "self" is used. So there should not be a difference from calling the constructor of the type vs having an ordinary function with the return value being the record.

Title: Re: Smart pointers revisited. Let me know what you think.
Post by: alpine on January 25, 2022, 04:32:23 pm

Quote from: Warfley on January 25, 2022, 02:50:11 pm

*snip*
The thing is, the constructor should be nothing more than a static class function where instead of Result "self" is used. So there should not be a difference from calling the constructor of the type vs having an ordinary function with the return value being the record.

Exactly! In case of a managed object the memory is (should be) already pre-allocated, so when the type name is used instead of real variable, the compiler makes a hidden temporary and then the issue manifests itself. See highlighted lines 34-36, 47-49, 50-52. Lines 34-36 finalizes the temporary at ebp-56 before constructor invocation (line 39) and the actual copy (line 43):

Code: ASM [Select][+]

.section .text.n_p$project2_$$_test1,"x"
        .balign 16,0x90
.globl  P$PROJECT2_$$_TEST1
P$PROJECT2_$$_TEST1:
# Temps allocated between ebp-56 and ebp-8
# [53] begin
        pushl   %ebp
        movl    %esp,%ebp
        leal    -56(%esp),%esp
        pushl   %ebx
# Temp -8,8 allocated
# Var R located at ebp-8, size=OS_64
        movl    $INIT_$P$PROJECT2_$$_TSOMEMANAGEDRECORD,%eax
        movl    %eax,%edx
        leal    -8(%ebp),%eax
        call    fpc_initialize
        leal    -56(%ebp),%eax
        movl    $INIT_$P$PROJECT2_$$_TSOMEMANAGEDRECORD,%edx
        call    fpc_initialize
# Temp -20,12 allocated
# Temp -44,24 allocated
# Temp -48,4 allocated
        movl    $1,%eax
        leal    -44(%ebp),%edx
        leal    -20(%ebp),%ecx
        call    fpc_pushexceptaddr
        call    fpc_setjmp
        pushl   %eax
        testl   %eax,%eax
        jne     .Lj16
# [54] R := TSomeManagedRecord.Create(42);
        movl    $INIT_$P$PROJECT2_$$_TSOMEMANAGEDRECORD,%ebx
# Temp -56,8 allocated
        leal    -56(%ebp),%eax
        movl    $INIT_$P$PROJECT2_$$_TSOMEMANAGEDRECORD,%edx
        call    fpc_finalize
        movl    $42,%edx
        leal    -56(%ebp),%eax
        call    P$PROJECT2$_$TSOMEMANAGEDRECORD_$__$$_CREATE$LONGINT$$TSOMEMANAGEDRECORD
        leal    -56(%ebp),%eax
        leal    -8(%ebp),%edx
        movl    %ebx,%ecx
        call    fpc_copy_proc
.Lj16:
        call    fpc_popaddrstack
# [55] end;
        leal    -56(%ebp),%eax
        movl    $INIT_$P$PROJECT2_$$_TSOMEMANAGEDRECORD,%edx
        call    fpc_finalize
        movl    $INIT_$P$PROJECT2_$$_TSOMEMANAGEDRECORD,%edx
        leal    -8(%ebp),%eax
        call    fpc_finalize
        popl    %eax
        testl   %eax,%eax
        je      .Lj15
        call    fpc_reraise
# Temp -44,24 released
# Temp -20,12 released
# Temp -48,4 released
.Lj15:
# Temp -8,8 released
        popl    %ebx
        movl    %ebp,%esp
        popl    %ebp
        ret

Same with the ordinary function (without using the result value).
Same with the enumerator example - the enumerator is held in a hidden temporary during the loop.

Title: Re: Smart pointers revisited. Let me know what you think.
Post by: Thaddy on January 25, 2022, 04:51:49 pm

And that is why I think that atm the interface approach works better. No temporaries.
But that can change...

Title: Re: Smart pointers revisited. Let me know what you think.
Post by: alpine on January 25, 2022, 05:08:17 pm

Quote from: Thaddy on January 25, 2022, 04:51:49 pm

And that is why I think that atm the interface approach works better. No temporaries.
But that can change...

Actually the Auto<> is also an advanced record and the issue is there. The good thing is perhaps that the subfields of the record are managed a bit deeper and a bit better by the compiler ... and this circumvents the asymmetrical Initialize/Finalize calls (no need to use them).

P.S. It seems that putting a TInterfacedObject into can be a viable workaround for the enumerator issue also.

Title: Re: Smart pointers revisited. Let me know what you think.
Post by: Thaddy on January 25, 2022, 06:02:54 pm

Well, let's see what ASerge comes up with. This is still relevant and - I think you all agree - a total joy to explore.

Title: Re: Smart pointers revisited. Let me know what you think.
Post by: Warfley on January 25, 2022, 06:08:33 pm

Quote from: y.ivanov on January 25, 2022, 05:08:17 pm

P.S. It seems that putting a TInterfacedObject into can be a viable workaround for the enumerator issue also.

Maybe, but as enumerators are freed automatically anyway, I can simply use classes. I just wanted to try out if records might be more efficient (as they do not need to allocate memory), so there is no real advantage of using interfaces over classes

Title: Re: Smart pointers revisited. Let me know what you think.
Post by: Thaddy on January 25, 2022, 06:25:03 pm

Quote from: Warfley on January 25, 2022, 06:08:33 pm

so there is no real advantage of using interfaces over classes

You missed the whole point about smart pointers: this can not be done with - pure - classes: automatic memory management (A.K.A. ARC, good or bad, no opinion). Automatic create on reference and automatic release....
You also seem to not fully understand interfaces and its use.. (which I find unlikely, since your contributions to the discussion, which are very valuable).
Summary:
When a class, instantiated through one of its supported interfaces and assigned to that interface variable, once the interface reference becomes nill, goes out of scope, the class will be destroyed automatically. (except for CORBA, but that is a relic/dinosour, nobody uses it apart from some forum member who wrote a brilliant - free - book about FPC/Lazarus and completely missed the point in his chapter about interfaces.... He knows my opinion to be fact...)

To me this thread is more like an academic discussion on how to achieve the best possible way to implement the topic.
Academic discussions often lead to application of such theory.
Especially the insights of ASerge are very valuable too. This is not a two way street, any suggestions and improvements (with proof!) are welcome.

Title: Re: Smart pointers revisited. Let me know what you think.
Post by: Warfley on January 25, 2022, 10:28:01 pm

I know, this was about enumerators, where the double finalize bug also occurs. Not generally for SmarPointers

Title: Re: Smart pointers revisited. Let me know what you think.
Post by: SymbolicFrank on January 26, 2022, 09:59:22 am

Half the reason this is an issue, is because Free doesn't nil the value. You have to use FreeAndNil consequently everywhere if you want to be able to see if a variable is freed or not. Why is that?

Another question: what would be the simplest way to make a single, specific class do automatic memory management? The one I have in mind has two constructors, both with parameters (I made the inherited ones private).

Title: Re: Smart pointers revisited. Let me know what you think.
Post by: Thaddy on January 26, 2022, 12:33:19 pm

FreeAndNil almost always simply hides bad programming. You should almost never need it in a well written program. The fpc/lazarus community seems to favor it, The Delphi community seems to use it with more caution.

There are several ways to do automatic memory management for classes, but not easily globally.
1. If applicable create the classes on the stack, I have given an example on this forum. That is very, very fast too.
2. Use my interface based Auto<T> , not any other for now. This may be different in the future as per this discussion.
3. Derive your classes from TInterfacedObject and only use interface references to create your objects.
4. Use a garbage collecting memory manager (included in ./packages/libgc ) made by Florian, and me?. I don't know how much he used from me, I merely alerted him and provided example code. He may have not used any of my code, which in turn was a FPC version of Barry Kelly's code.

It is interesting to know that Delphi dropped ARC, which is a form of global automatic memory management that does not rely on garbage collection.

Title: Re: Smart pointers revisited. Let me know what you think.
Post by: SymbolicFrank on January 27, 2022, 09:51:23 am

Quote from: Thaddy on January 26, 2022, 12:33:19 pm

FreeAndNil almost always simply hides bad programming. You should almost never need it in a well written program. The fpc/lazarus community seems to favor it, The Delphi community seems to use it with more caution.

Well, it depends. Like, if you want the intersection of two StringLists:

Code: Pascal [Select][+]

function Both(List1, List2: TStringList): TStringList;

Obvious and easy, right? But now there are three objects that have to be freed at some point. It's clear the caller has to do that, right? But that's not where it is created. So, inserting the matching Free immediately afterwards typing the Create doesn't work.

Code: Pascal [Select][+]

function Both(List1, List2: string): string;

How about this? Use the TStringList.Text property. But that requires strings without line breaks. And there is more overhead.

Code: Pascal [Select][+]

function Both(List1, List2: TStringArray): TStringArray;

How about this? Still overhead, but at least you don't have to Free them.

Code: Pascal [Select][+]

function Both(List1, List2: TStringList; out These: TStringList): Boolean;

Many people think this is the way to do it. That way, the caller has to Create and Free all of them. I sometimes do this, if it is important to return a clear status, but I really don't like it. Useful in C++ style where everything generates an exception, but not in Borland style, where nil or Count = 0 are used.

If you have a class that has a property that is another class, it should be obvious that you shouldn't free it outside the encapsulating class. But you could. And it is not easy to make a copy, so you keep passing around pointers to that same class instance, or make more of the encapsulating one. Many also allow you to assign those property classes. So when do you free what if you assign the property of one to another?

Quote

It is interesting to know that Delphi dropped ARC, which is a form of global automatic memory management that does not rely on garbage collection.

I vastly prefer refcounting to garbage collection, because it keeps memory usage in check and doesn't periodically freeze your application. But I'll see if I can use the TInterfacedObject.

Title: Re: Smart pointers revisited. Let me know what you think.
Post by: Warfley on January 27, 2022, 11:59:42 am

Well, this is actually quite easy, when you have a function that returns a new object you to handle the function call as create, but free the Result in error case in the function

Code: Pascal [Select][+]

function Both(List1, List2: TStringList): TStringList;
begin
  ...
  Result := TStringList.Create;
  try
    ...
  except
    Result.Free;
    raise;
  end;
end;
 
// usage:
  UnionList := Both(ListA, ListB);
  try
    ...
  finally
    UnionList.Free;
  end;

That said, there shouldnt't be a reason one has to care for this stuff in the first place. This is very easy to get wrong, and automatic memory management is completely fine for most cases. I wish we could override the ^ operator, this would allow to create smartpointers accessible with the already existing pointer syntax

Title: Re: Smart pointers revisited. Let me know what you think.
Post by: Bogen85 on September 25, 2022, 07:21:47 am

Thaddy, your smart pointer proof of concept is very cool.

I tried it with a few of my existing object pascal classes that I've written, and it appears to work very well and work as expected.

The destructors don't appear to be called when smart pointers are used in a program's begin ... end. block but they do get called when used in a separate procedure, which is fine.

However, I've only gotten classes to work that don't have arguments in their constructor.
I got around this by making a generic class that wraps one pointer and frees it in it's destructor.

But I'd prefer to have the smart pointer record handle that.

Regardless though, this is very promising (for someone coming from C++ to free pascal).

EDIT: (Well, coming back to Pascal after not using it for a couple decades, and extensively using several other programming languages in the interim)

Title: Re: Smart pointers revisited. Let me know what you think.
Post by: ASerge on September 25, 2022, 02:18:21 pm

Quote from: Thaddy on January 25, 2022, 06:02:54 pm

Well, let's see what ASerge comes up with. This is still relevant and - I think you all agree - a total joy to explore.

Sorry, I somehow missed this conversation.
The problem with sudden double release is easily solved - I forgot to implement the AddRef method.
So the updated unit

Code: Pascal [Select][+]

unit smartptrs;
 
{$MODE OBJFPC}
{$MODESWITCH ADVANCEDRECORDS}
 
interface
 
type
  generic TAuto<T:class> = record
  strict private type
    TRef = record Item: T; RefCount: LongInt; end;
    PRef = ^TRef;
  strict private
    FRef: PRef;
    class operator AddRef(var Self: TAuto);
    class operator Copy(constref Src: TAuto; var Dst: TAuto);
    class operator Finalize(var Self: TAuto); inline;
    class operator Initialize(var Self: TAuto); inline;
    function GetInstance: T; inline;
    procedure Release;
    procedure SetInstance(const Value: T);
  public
    class operator Explicit(constref From: TAuto): T; inline;
    property Instance: T read GetInstance write SetInstance;
  end;
 
implementation
 
class operator TAuto.AddRef(var Self: TAuto);
begin
  if Assigned(Self.FRef) then
    InterlockedIncrement(Self.FRef^.RefCount);
end;
 
class operator TAuto.Copy(constref Src: TAuto; var Dst: TAuto);
begin
  Dst.Release;
  if Assigned(Src.FRef) then
    InterlockedIncrement(Src.FRef^.RefCount);
  Dst.FRef := Src.FRef;
end;
 
class operator TAuto.Explicit(constref From: TAuto): T;
begin
  Result := From.Instance;
end;
 
class operator TAuto.Finalize(var Self: TAuto);
begin
  Self.Release;
end;
 
class operator TAuto.Initialize(var Self: TAuto);
begin
  Self.FRef := nil;
end;
 
function TAuto.GetInstance: T;
begin
  Result := nil;
  if Assigned(FRef) then
    Result := FRef^.Item;
end;
 
procedure TAuto.Release;
begin
  if Assigned(FRef) then
  begin
    if InterlockedDecrement(FRef^.RefCount) = 0 then
    begin
      FRef^.Item.Free;
      Dispose(FRef);
    end;
    FRef := nil;
  end;
end;
 
procedure TAuto.SetInstance(const Value: T);
begin
  if Assigned(FRef) then
  begin
    if Value = FRef^.Item then
      Exit;
    Release;
  end;
  if Assigned(Value) then
  begin
    New(FRef);
    FRef^.Item := Value;
    FRef^.RefCount := 1;
  end;
end;
 
end.

and the updated Test

Code: Pascal [Select][+]

{$APPTYPE CONSOLE}
{$IFDEF FPC}{$MODE OBJFPC}{$ENDIF}
 
uses smartptrs;
 
type
  TTest = class
  private
    FValue: Integer;
  public
    constructor Create(AValue: Integer); overload;
    property Value: Integer read FValue write FValue;
  end;
 
constructor TTest.Create(AValue: Integer);
begin
  FValue := AValue;
end;
 
type
  TAutoTest = specialize TAuto<TTest>;
var
  GlobRef: TAutoTest;
 
procedure TestCopyOnWrite;
var
  LocalRef: TAutoTest;
begin
  WriteLn('> TestCopyOnWrite');
  LocalRef.Instance := TTest.Create(-1);
  WriteLn('TTest(LocalRef).Value = ', TTest(LocalRef).Value);
  TTest(LocalRef).Value := 101;
  WriteLn('TTest(LocalRef).Value = ', TTest(LocalRef).Value);
  Writeln('Copy Local to Global and release it');
  GlobRef := LocalRef;  // TTest(GlobRef).Value = 101
  LocalRef.Instance := nil; //early release
  Writeln('Create new Local');
  LocalRef.Instance := TTest.Create(0);
  WriteLn('TTest(LocalRef).Value = ', TTest(LocalRef).Value);
  WriteLn('< TestCopyOnWrite');
end;
 
procedure TestChangeInternalField(R: TAutoTest);
begin
  WriteLn('> TestChangeInternalField');
  R.Instance.Value := -2;
  WriteLn('< TestChangeInternalField');
end;
 
begin
  GlobRef.Instance := TTest.Create(-1);
  TestChangeInternalField(GlobRef);
  WriteLn;
  WriteLn('TTest(GlobRef).Value = ', TTest(GlobRef).Value);
  WriteLn;
  TestCopyOnWrite;
  WriteLn;
  WriteLn('TTest(GlobRef).Value = ', TTest(GlobRef).Value);
  WriteLn;
  ReadLn;
end.

Title: Re: Smart pointers revisited. Let me know what you think.
Post by: Bogen85 on September 25, 2022, 08:44:31 pm

Quote from: ASerge on September 25, 2022, 02:18:21 pm

Sorry, I somehow missed this conversation.
The problem with sudden double release is easily solved - I forgot to implement the AddRef method.
So the updated unit...

Thanks ASerge, I believe your updated unit addresses some of my questions as well. I'll try to use it.

Title: Re: Smart pointers revisited. Let me know what you think.
Post by: Bogen85 on September 25, 2022, 10:58:32 pm

Well, I'll need to study your update (TAuto) more ASerge.

I like it how I can set the instance pointer directly with your example.

It works, but I have to do more checking to prevent Access violations compared to Thaddy's example (Auto)

Overall I like how Auto works, but I'd prefer to have a way to use procedure/property with it to set the instance pointer as well directly from a constructed object so that I don't need to have an additional pointer wrapper to allow for that.

I see that TAuto is using a record finalize member which I assume is a akin to a destructor, where as Auto is using a destructor in the nested TFreeTheValue to accomplish something similar.

I just need to learn advanced records and IInterface and TInterfacedObject ...

Title: Re: Smart pointers revisited. Let me know what you think.
Post by: Bogen85 on October 05, 2022, 07:21:02 pm

Quote from: Bogen85 on September 25, 2022, 10:58:32 pm

Well, I'll need to study your update (TAuto) more ASerge.

I like it how I can set the instance pointer directly with your example.

It works, but I have to do more checking to prevent Access violations compared to Thaddy's example (Auto)

I was using Thaddy's Auto for a while but I had to wrap classes with parameterized creates in a separate pointer holder.
That was working but had added overhead was a bit awkward.

After getting use to doing it that way I revisited ASerge's and was able to resolve all my access/initialization problems.

I understand ASerge's TAuto unit a lot more than Thaddy's Auto unit.

Also, it is simpler for me to use. I just had learn a few things using Auto before I revisited TAuto.

Thanks ASerge! ;D

Title: Re: Smart pointers revisited. Let me know what you think.
Post by: SymbolicFrank on October 10, 2022, 03:01:20 pm

Ok, let's make an overview. Uses objpash.inc.

We have classes, that have a virtual method table (record TVmt), static methods and class methods, like:

Code: Pascal [Select][+]

class function NewInstance: TObject; virtual;

If you call the constructor, NewInstance is called in the background.

The difference between class methods and other methods is, that the latter get an invisible first parameter, Self, that points to the instance. While class methods are regular functions. NewInstance is just a function that returns a TObject.

After the TObject, we see the definition of IUnknown. This basically extends your classes with two new functions: _AddRef and _Release, which are used for reference counting.

The basic class that implements both, is TInterfacedObject. The main difference with a TObject is, that it adds a property RefCount and overrides NewInstance and Destroy, to add to or substract from that reference count.

Why does it do that and how does that help us with the memory management?

How it actually works isn't explained in this source file. To understand that, we have to read between the lines and fill in the blanks that are hidden inside the compiler. Like. how it calls NewInstance automatically. Every time you assign an instance to a variable, the compiler calls _AddRef (if available). And every time you assign something else to that variable, or it goes out of scope, it calls _Release. See:

Code: Pascal [Select][+]

function TObject.GetInterface(const iid : tguid; out obj) : boolean;

(For the implementations, see 'function fpc_intf*' in compproc.inc. It's hard to show the code where this is used, because the compiler uses offsets into the VMT instead of function names to call the _AddRef and _Release methods. To see it happen, use the debugger.)

The result of all that is, that if there are no variables anymore that reference an instance, it is deleted.

Or, in other words: instead of inheriting from class(TObject), use class(TInterfacedObject) to create your own ones and call the inherited destructor as usual. Like this:

Code: Pascal [Select][+]

type
  TMyClass = class(TInterfacedObject)
    MyVar: string;
    constructor Create(ThisVar: string); virtual;
    destructor Destroy; override;
  end;
 
constructor TMyClass.Create(ThisVar: string);
begin
  inherited Create;
  MyVar := ThisVar;
end;
 
destructor TMyClass.Destroy;
begin
  inherited Destroy;
end;
 
var
  m, n: TMyClass;
begin
  m := TMyClass.Create('Bla');
  n := m;
  m := TMyClass.Create('Zomg!');
  // Do stuff
  // Don't Free them!
end;

That should be enough.

But, there's a lot of existing classes you didn't write yourself, where that doesn't work. What do you do?

Well, the easiest way is probably to inherit a new class, that adds and implements an IUnknown interface, like TInterfacedObject does. Or use a smart pointer. The most interesting one is probably this one as posted earlier (https://adugmembers.wordpress.com/2011/12/05/smart-pointers/), which should now work on trunk.

Title: Re: Smart pointers revisited. Let me know what you think.
Post by: Bogen85 on October 10, 2022, 03:52:43 pm

Quote from: SymbolicFrank on October 10, 2022, 03:01:20 pm

But, there's a lot of existing classes you didn't write yourself, where that doesn't work. What do you do?

Well, the easiest way is probably to inherit a new class, that adds and implements an IUnknown interface, like TInterfacedObject does. Or use a smart pointer. The most interesting one is probably this one as posted earlier (https://adugmembers.wordpress.com/2011/12/05/smart-pointers/), which should now work on trunk.

I'll need to look at that one. I'm using a very short class function (a constructor wrapper) I added to TAuto instantiate existing classes fairly seamlessly, and that has been working well.

I need to post my TAuto additions and slight modifications (does not change it's core functionality in any way, and TAuto only provided the core functionality anyways) but I'm not ready to do that quite yet.

Title: Re: Smart pointers revisited. Let me know what you think.
Post by: PascalDragon on October 11, 2022, 07:32:01 am

Quote from: SymbolicFrank on October 10, 2022, 03:01:20 pm

The difference between class methods and other methods is, that the latter get an invisible first parameter, Self, that points to the instance. While class methods are regular functions.

Wrong. There are normal class methods and there are static class methods. The later are always marked with a static modifier at the end. Those (and only those) behave like regular functions or procedures that just happen to be inside a class scope and can be assigned to normal function pointers (at least in main, I think 3.2.x didn't allow that yet).
Regular class methods however do also have that hidden Self parameter, but it won't contain the class instance, but the class type. Regular class methods can also be virtual which means that a sub class can override them. This makes them a very powerful feature (something I deeply miss here and then in C++) and is the reason why NewInstance can be reimplemented by sub classes to use some different allocation mechanism.

Title: Re: Smart pointers revisited. Let me know what you think.
Post by: SymbolicFrank on October 11, 2022, 10:00:14 am

Thanks for making that clear :)

I don't really use that very much.

Title: Re: Smart pointers revisited. Let me know what you think.
Post by: Ryan J on October 16, 2022, 04:49:31 am

Quote from: Thaddy on August 07, 2019, 07:37:46 am

The [box] syntax originated from the current lack of default properties other that array properties.

I just stumbled across this post and thought I'd share my feature branch at https://gitlab.com/genericptr/free-pascal/-/tree/default_record_property so maybe one day some progress can be made on this and we don't forget about it. Sven (@PascalDragon I think?) agreed that smart pointers are the best option for ARC in FPC and I think approved of a default property If I recall correctly so I made this branch which implements a default property for records with a class field.

If the compiler devs ever want to try to implement I'll come back to it but it's just parked here for now.

Title: Re: Smart pointers revisited. Let me know what you think.
Post by: Thaddy on October 16, 2022, 10:26:13 am

That is already recorded.
Both AVK and me wrote a proper implementation. Just waiting for a water tight implementation for the default property, which was more or less also implemented in a fork by Maciej. (and as far as I know almost correct, but abandoned?)

Both mentioned implementations work from 3.2 onwards. It is just the sugar candy that does not work yet.

Look up the discussion on this forum.

Title: Re: Smart pointers revisited. Let me know what you think.
Post by: Ryan J on October 17, 2022, 03:58:47 am

Quote from: Thaddy on October 16, 2022, 10:26:13 am

That is already recorded.
Both AVK and me wrote a proper implementation. Just waiting for a water tight implementation for the default property, which was more or less also implemented in a fork by Maciej. (and as far as I know almost correct, but abandoned?)

Both mentioned implementations work from 3.2 onwards. It is just the sugar candy that does not work yet.

Look up the discussion on this forum.

I'm sorry, there is already a default property developed? Can you show me where it is so I can study it? I didn't get any guidance from Sven on this so I have no idea what would be allowed or not or if I implemented it correctly.

The one I did was pretty complete I think (including for..in loops) but it's scoped was constrained to just class fields since using other types affect far too much of the compiler and was likely to get rejected anyways.

Title: Re: Smart pointers revisited. Let me know what you think.
Post by: Thaddy on October 17, 2022, 11:14:52 am

Maciej, a.k.a HNB on this forum did it. He is not good with the compiler people anymore.
(But he contributed many things to Lazarus and FPC: anchor docking resp rtl-generics and many fixes)

Title: Re: Smart pointers revisited. Let me know what you think.
Post by: Ryan J on October 18, 2022, 08:38:52 am

Quote from: Thaddy on October 17, 2022, 11:14:52 am

Maciej, a.k.a HNB on this forum did it. He is not good with the compiler people anymore.
(But he contributed many things to Lazarus and FPC: anchor docking resp rtl-generics and many fixes)

yes I spoke with him around the time he did the excellent record management operators and we discussed the default property then. At that time he hadn't implement it so I assumed it was left undone. Not sure where I can find his implementation anyways and since he's not going to contribute to FPC anymore it may be hard to get what he has working or finished.

Honestly even if the compiler devs accept this it will be years before it's reviewed and merged in. Personally I have an extended RTTI patch which is in the pipeline for review but it's been well over a year since I hard back from them on that so I expect many years for the default property unfortunately.

Title: Re: Smart pointers revisited. Let me know what you think.
Post by: Bogen85 on February 04, 2023, 05:06:00 pm

I started a continuation of this topic/thread here: https://forum.lazarus.freepascal.org/index.php/topic,62163.0.html
As this thread is over 3 months old, please continue any discussion on the above topic/thread.

Title: Re: Smart pointers revisited. Let me know what you think.
Post by: circular on February 04, 2023, 06:54:00 pm

I find that often I just need my local variables to be freed, without reference counting. And ideally I would like not to add any more code than with arrays.

Here is a record to handle that for classes having a constructor without parameters:

Code: Pascal [Select][+]

unit Local;
 
{$modeswitch advancedRecords}
 
interface
 
type
  { LocalClass }
 
  generic LocalClass<T: class> = record
  private
    type TSelf = specialize LocalClass<T>;
    var FInstance: T;
    class operator Initialize(var ASelf: TSelf); inline;
    class operator Finalize(var ASelf: TSelf); inline;
    function GetInstance: T; inline;
    procedure SetInstance(const AInstance: T); inline;
  public
    function Give: T;
    property _: T read GetInstance write SetInstance;
  end;
 
implementation
 
class operator LocalClass.Initialize(var ASelf: TSelf);
begin
  ASelf.FInstance:= nil;
end;
 
class operator LocalClass.Finalize(var ASelf: TSelf);
begin
  ASelf.FInstance.Free;
end;
 
procedure LocalClass.SetInstance(const AInstance: T);
begin
  if Assigned(FInstance) then
     FInstance.Free;
  FInstance := AInstance;
end;
 
function LocalClass.GetInstance: T;
begin
  if FInstance = nil then
     FInstance := T.Create;
  result := FInstance;
end;
 
function LocalClass.Give: T;
begin
  result := FInstance;
  FInstance := nil;
end;
 
end.

It can be used like that:

Code: Pascal [Select][+]

uses Classes, Local;
 
type
  StringList = specialize LocalClass<TStringList>;
 
var
  list: StringList;
  taken: TStringList;
 
begin
  list._.Add('Hello');
  list._.Add('Alice');
  list._.Sort;
  WriteLn(list._.CommaText);
 
  taken := list.Give;
  // display 0 because list is reset
  writeln(list._.Count);
 
  taken.Add('Bob');
  // display Alice,Bob,Hello
  WriteLn(taken.CommaText);
  taken.Free; // need to free it
 
  list._.Add('Alice');
  // clear the list
  list._ := nil;
  list._.Add('Mirror');
  // display Mirror
  WriteLn(list._.CommaText);
end.