Print Page - Strange bugs with Advanced Records

Free Pascal => General => Topic started by: ad1mt on October 28, 2024, 08:15:52 pm

Title: Strange bugs with Advanced Records - SOLVED
Post by: ad1mt on October 28, 2024, 08:15:52 pm

EDIT: This problem has been solved... see my later post.
I'm getting strange bugs using advanced records with methods and a dynamic array inside.
In the following scenario...

Code: Pascal [Select][+]

type R=record {various bits'n'bobs} end;
var u,v:R;
procedure P(const p1:R; var p2:R); begin p2:=0 end;
begin
u:= 1;v:= u;
P(v);
// after the procedure call, the u variable also gets the value zero!
 

...then after the procedure call, both the u & v variables get overwritten. This happens to the p1 and p2 parameters inside the procedure, and in u & v variables outside the procedure. And it happens, even though the u variable is passed to the procedure as a const parameter.
NB this problem is very intermittent; the above code is "example" code, and not guaranteed to run or produce the problem.
EDIT: I forgot to say that if the procedure is defined with an out parameter instead of a var parameter like this, then the problem disappears:

Code: Pascal [Select][+]

procedure P(const p1:R; out p2:R);

I have made some progress: it apears that when the assignment v:=u is done, the compiler sometimes copies a reference and not the record data. Then, when the v variable is accessed inside the procedure, it also overwrites the u variable.
I have worked around the problem, by creating a procedure that explicitly copies one record to another, e.g.

Code: Pascal [Select][+]

procedure copy_R(var P1,P2);begin P2.field:= P1.field; etc;end;

and use that instead of assignments like v:=u;
The problem is that I am creating a library, and if users of my library also hit the same problem, then they are going to get very strange bugs that will be impossible to fix.
I can't post my actual code because there is too much. I will have another attempt to try to produce some minimal code that shows the problem, and if successful I will post a follow-up.
I've been battling with this problem for many months now, and would welcome any suggestions. I have searched the internet but didn't get any good results. I am getting desperate.
Thanks.

Title: Re: Strange bugs with Advanced Records
Post by: marcov on October 28, 2024, 08:23:13 pm

If a problem is hard to reduce it might depend on somewhere else that is corrupting memory.

Anyway, reducing until reproduction is the only way.

Title: Re: Strange bugs with Advanced Records
Post by: Fibonacci on October 28, 2024, 08:24:13 pm

constref

Title: Re: Strange bugs with Advanced Records
Post by: WooBean on October 28, 2024, 08:34:14 pm

Quote

I've been battling with this problem for many months now, and would welcome any suggestions.

Give us a chance to help you. Do what Marcov asked.

My guess is that problem lies in {various bits'n'bobs} and

Code: Pascal [Select][+]

  u:= 1; // this is an opposite to what a record (advanced also) can accept.
 

Title: Re: Strange bugs with Advanced Records
Post by: Bart on October 28, 2024, 10:23:40 pm

Pleas show some compileable code.

Bart

Title: Re: Strange bugs with Advanced Records
Post by: Joanna on October 28, 2024, 11:25:55 pm

I’m confused as well. How can you set a record to 1? Shouldn’t it be a field inside the record set to 1?

Title: Re: Strange bugs with Advanced Records
Post by: jamie on October 29, 2024, 12:23:00 am

The real issue is that incomplete example of the Record definition leads everyone a stray.

Since "Advanced Records" was mentioned, I would venture to say that Operators for ":=" could be in use, this would allow to
assign a "0" to a record where the interaction is taking place within the OPERATOR := code body.

Just my guess and I am sticking to it! :D

Title: Re: Strange bugs with Advanced Records
Post by: Joanna on October 29, 2024, 12:28:10 am

Interesting how does that work?

Title: Re: Strange bugs with Advanced Records
Post by: jamie on October 29, 2024, 12:42:32 am

Glad you ask! :D

Code: Pascal [Select][+]

unit Unit1;
 
{$mode objfpc}{$H+}
{$modeSwitch AdvancedRecords}
 
interface
 
uses
  Classes, SysUtils, Forms, Controls, Graphics, Dialogs, StdCtrls;
 
type
 
 { TMyRecord }
 
 TMyRecord = Record
   P:Integer;
   class operator := (ConstRef A:TmyRecord):integer;
   Class operator := (B:Integer):TMyRecord;
 end;
 
  { TForm1 }
 
  TForm1 = class(TForm)
    Button1: TButton;
    procedure Button1Click(Sender: TObject);
  private
 
  public
 
  end;
 
var
  Form1: TForm1;
 
implementation
 
{$R *.lfm}
 
{ TMyRecord }
 
class operator TMyRecord.:=(ConstRef A: TmyRecord): integer;
begin
  Result := A.P;
end;
 
class operator TMyRecord.:=(B: Integer): TMyRecord;
begin
  Result.P := B;
end;
 
{ TForm1 }
 
procedure TForm1.Button1Click(Sender: TObject);
Var
  M:TMyRecord;
  I:integer;
begin
  M:= 1000;
  I:= M;
  Caption := I.ToString;
end;
 
end.
 
 

Feast your eyes on that magic code! :-*

Title: Re: Strange bugs with Advanced Records
Post by: Joanna on October 29, 2024, 01:20:07 am

Thanks for the code. Those overloaded operators are confusing :D
I presume they are always like functions with a result?
I remember seeing something like this in c++ a long time ago. I guess it does make a good looking code using := .

Title: Re: Strange bugs with Advanced Records
Post by: jamie on October 29, 2024, 01:30:13 am

Quote

Why not use a property for setting values of record fields?

Well, that would make it more readable! what's the matter for you! :)

Title: Re: Strange bugs with Advanced Records
Post by: Joanna on October 29, 2024, 01:32:53 am

It just took me a moment to understand it. So it’s basically the same as doing record.field:= value. The only use case I can think of is if you want to set several fields to a given value.

Title: Re: Strange bugs with Advanced Records
Post by: jamie on October 29, 2024, 02:10:48 am

My last cryptic code pieace! :D

Code: Pascal [Select][+]

unit Unit1;
 
{$mode objfpc}{$H+}
{$modeSwitch AdvancedRecords}
 
interface
 
uses
  Classes, SysUtils, Forms, Controls, Graphics, Dialogs, StdCtrls;
 
type
 
 { TMyRecord }
 
 TMyRecord = Record
   P:Integer;
   X,Y:integer;
   Function fGEtSetXY(inX,InY:Integer):TMyRecord;
   class operator := (ConstRef A:TmyRecord):integer;
   Class operator := (B:Integer):TMyRecord;
   Property XY[iX,iY:Integer]:TMyRecord Read FGetSetXY; default;
 end;
 
  { TForm1 }
 
  TForm1 = class(TForm)
    Button1: TButton;
    procedure Button1Click(Sender: TObject);
  private
 
  public
 
  end;
 
var
  Form1: TForm1;
 
implementation
 
{$R *.lfm}
 
{ TMyRecord }
 
function TMyRecord.fGEtSetXY(inX, InY: Integer): TMyRecord;
begin
 X := inX;Y:=inY;
Result := Self;
end;
 
class operator TMyRecord.:=(ConstRef A: TmyRecord): integer;
begin
  Result := A.P;
end;
 
class operator TMyRecord.:=(B: Integer): TMyRecord;
begin
  Result.P := B;
end;
 
{ TForm1 }
 
procedure TForm1.Button1Click(Sender: TObject);
Var
  M:TMyRecord;
  I:integer;
begin
  M[100,200]; //Set the record's X,Y as a default;
  M:= 1000;   //Set P variable;
  I:= M;      // Assign I from M.P
  Caption := I.ToString+':'+M.X.Tostring+','+M.Y.Tostring;
end;
 
end.
 
 

That Default X,Y setting parameter should really spin your head around!

:o

Title: Re: Strange bugs with Advanced Records
Post by: Joanna on October 29, 2024, 02:49:57 am

I didn’t even know that constref existed before.
https://wiki.freepascal.org/Constref
Is this is for multi threaded programs?

I’ve never used default before I probably should.
https://wiki.freepascal.org/Default

That is some interesting code :)

Title: Re: Strange bugs with Advanced Records
Post by: 440bx on October 29, 2024, 04:27:53 am

Quote from: Joanna from IRC on October 29, 2024, 02:49:57 am

I didn’t even know that constref existed before.
https://wiki.freepascal.org/Constref
Is this is for multi threaded programs?

"constref" has nothing to do with multi-threaded programs. It is a way to tell the compiler to pass the parameter by reference and that in spite of it being passed by reference, the parameter cannot be modified, i.e, is constant. "constref" = constant by reference (reference = pass the address of the structure.)

It is normally used when passing large data structures that the routine only needs to read (will not modify.) For instance, passing an array of 1,000,000 elements to a function/procedure is best done as "constref", otherwise the compiler will create a _copy_ of the array on the stack (if neither "var" nor "const" nor "constref" are specified as parameter modifiers.)

Note that the documentation states that the programmer should _not_ assume that a "const" parameter will be passed by reference, which is correct because if the parameter fits in a register then it can simply be placed in a register, whereas if the parameter is a structured type then "const" will cause its address to be passed to the routine. "constref" makes it clear that it will always be passed by reference (unlike "const" which may or may not pass by reference depending on the size of the argument.)

HTH.

Title: Re: Strange bugs with Advanced Records
Post by: Joanna on October 29, 2024, 06:33:55 am

I thought it might be for multi threaded programs because of

Quote

This qualifier informs the compiler that within the entire program there is no code that will change the value of the parameter while the procedure/function is executing.
This means that not only the parameter, but also the variable (in above example: x) passed by the caller (e.g. a global var) must not be changed until the call with the constref parameter has returned.

Title: Re: Strange bugs with Advanced Records
Post by: Fibonacci on October 29, 2024, 07:13:29 am

Quote

This qualifier informs the compiler that within the entire program there is no code that will change the value of the parameter while the procedure/function is executing.

That is so unbeliveable I had to test it. And it turns out to NOT be true. A variable CAN be modified while some other rountine is "using" the variable as a constref.

It must mean something else, or thats simply not true.

Code: Pascal [Select][+]

program app;
 
uses SysUtils;
 
var
  aint: integer = 0;
  cs: TRTLCriticalSection;
 
function thr1(p: pointer): ptrint;
var
  i: integer;
begin
  result := 0;
 
  // for 5 secs
  for i := 1 to 5 do begin
    // each 1 second modify "aint"
    sleep(1000);
    // in a critical section! extra protection
    EnterCriticalSection(cs);
    try
      write('thr1 modifying aint... ');
      aint += 1;
      write('value now = ', aint, '...');
      writeln;
    finally
      LeaveCriticalSection(cs);
    end;
  end;
end;
 
procedure proc1(constref x: integer);
var
  i, calc: integer;
begin
  writeln('proc1: simulating using the value');
  writeln('proc1: is @x = @aint? ', @x = @aint);
  calc := 0;
  // for 6 secds
  for i := 1 to 6 do begin
    // each 1 second use "x" which is "aint"
    sleep(1000);
    // "use" the "x"     
    // in a critical section! extra protection
    EnterCriticalSection(cs);
    try
      writeln('proc1: value of constref = ', x);
      calc += x;
    finally
      LeaveCriticalSection(cs);
    end;
  end;
  writeln('proc1: value of "calc" = ', calc, ', SHOULD BE 0 if other code couldnt modify the constref');  
  writeln('proc1: work done, constref should be released now');
end;
 
procedure main;
begin
  InitCriticalSection(cs);
  writeln('main: current value = ', aint);
  writeln('main: starting thr1 that will try to modify the value');
  writeln('main: ... each 1 second, first try after 1 second');
  BeginThread(@thr1, nil);
  sleep(100); // little sleep after thread start
  writeln('main: in the meantime call proc1 which will work in a loop for 6 seconds');
  writeln('main: calling proc1 (lock for 6 sec)');
  proc1(aint);
  writeln('main: proc1 done');
  writeln('main: current value = ', aint);
  writeln;
  DoneCriticalSection(cs);
  readln;
end;
 
begin
  main;
end.

Code: Text [Select][+]

main: current value = 0
main: starting thr1 that will try to modify the value
main: ... each 1 second, first try after 1 second
main: in the meantime call proc1 which will work in a loop for 6 seconds
main: calling proc1 (lock for 6 sec)
proc1: simulating using the value
proc1: is @x = @aint? TRUE
thr1 modifying aint... value now = 1...
proc1: value of constref = 1
thr1 modifying aint... value now = 2...
proc1: value of constref = 2
thr1 modifying aint... value now = 3...
proc1: value of constref = 3
thr1 modifying aint... value now = 4...
proc1: value of constref = 4
thr1 modifying aint... value now = 5...
proc1: value of constref = 5
proc1: value of constref = 5
proc1: value of "calc" = 20, SHOULD BE 0 if other code couldnt modify the constref
proc1: work done, constref should be released now
main: proc1 done
main: current value = 5

Title: Re: Strange bugs with Advanced Records
Post by: Fibonacci on October 29, 2024, 07:28:34 am

https://wiki.freepascal.org/Constref

Also what does this example code show? Nothing, what is it?

Switch to the French version, the truth is there.

Title: Re: Strange bugs with Advanced Records
Post by: TRon on October 29, 2024, 08:06:10 am

Quote from: Fibonacci on October 29, 2024, 07:28:34 am

Switch to the French version, the truth is there.

I rather keep it to the official documentation about constant parameters (https://www.freepascal.org/docs-html/ref/refsu67.html)

Title: Re: Strange bugs with Advanced Records
Post by: cdbc on October 29, 2024, 08:11:27 am

Quote

Also what does this example code show?

AFAICS it tries to show you, that you can call a /reference/ parameter(constref) with a literal value too...
Regards Benny

Title: Re: Strange bugs with Advanced Records
Post by: Thaddy on October 29, 2024, 09:03:12 am

It shows a border case where you can not change the parameter through T, but if T shares the same address as a global or accessible variable elsewhere, then the compiler assumes you want to modify through S instead and allows that.
It demonstrate that gotcha that is applicable to both const and constref, btw.
T is protected within the scope of the method, but S itself has higher scope and within that scope modification of S is allowed. It violates basically the contract beween compiler and programmer as described.

Title: Re: Strange bugs with Advanced Records
Post by: ad1mt on October 29, 2024, 09:30:52 am

This code demonstrates the problem...

Code: Pascal [Select][+]

{$MODE OBJFPC}
{$MODESWITCH ADVANCEDRECORDS}
program test_extended_record_5;
const
ISize=5;
IMax=4;
 
type
REC = record
        private
        I:array of integer;
        public
        class operator :=(const v1:integer):REC;
        end;
 
class operator REC.:=(const v1:integer):REC;
var     N:integer;
begin
setlength(Result.I,ISize);
for N:=0 to IMax do Result.I[N]:= v1;
end;
 
var N:integer;
 
procedure TrashREC(const P1:REC; var P2:REC);
begin
writeln('TrashREC');
P2.I[0]:= 100;
end;
 
procedure P;
var     U,V :REC;
begin
U:= 99;
V:= U;
write('U='); for N:=0 to IMax do write(U.I[N],' '); writeln;
write('V='); for N:=0 to IMax do write(V.I[N],' '); writeln;
TrashREC(U,V);
write('U='); for N:=0 to IMax do write(U.I[N],' '); writeln;
write('V='); for N:=0 to IMax do write(V.I[N],' '); writeln;
writeln;
end;
 
begin
P;
end.
 

After calling the TrashREC procedure, the first array element of U is also incorrectly overwritten, as follows:

U=99 99 99 99 99 V=99 99 99 99 99 TrashREC U=100 99 99 99 99 V=100 99 99 99 99
While investigating this, I found this page in the wiki...
https://wiki.freepascal.org/management_operators (https://wiki.freepascal.org/management_operators)
This gives an explanation of what is happening, and a solution to the problem. The fixed/working program is:

Code: Pascal [Select][+]

{$MODE OBJFPC}
{$MODESWITCH ADVANCEDRECORDS}
program test_extended_record_6;
const
ISize=5;
IMax=4;
 
type
REC = record
        private
        I:array of integer;
        public
        class operator :=(const v1:integer):REC;
        class operator copy(constref v1:REC; var V2:REC);
        end;
 
class operator REC.copy(constref v1:REC; var V2:REC);
var     N:integer;
begin
setlength(v2.I,ISize);
for N:=0 to IMax do v2.I[N]:= v1.I[N];
end;
 
class operator REC.:=(const v1:integer):REC;
var     N:integer;
begin
setlength(Result.I,ISize);
for N:=0 to IMax do Result.I[N]:= v1;
end;
 
var N:integer;
 
procedure TrashREC(const P1:REC; var P2:REC);
begin
writeln('TrashREC');
P2.I[0]:= 100;
end;
 
procedure P;
var     U,V :REC;
begin
U:= 99;
V:= U;
write('U='); for N:=0 to IMax do write(U.I[N],' '); writeln;
write('V='); for N:=0 to IMax do write(V.I[N],' '); writeln;
TrashREC(U,V);
write('U='); for N:=0 to IMax do write(U.I[N],' '); writeln;
write('V='); for N:=0 to IMax do write(V.I[N],' '); writeln;
writeln;
end;
 
begin
P;
end.
 

Note the use of constref with the v1 parameter of the copy procedure. If you use const instead, you get a compile error "Impossible operator overload". If anyone can explain why this is, I would be curious to know.

Title: Re: Strange bugs with Advanced Records
Post by: dbannon on October 29, 2024, 09:34:21 am

Quote from: Fibonacci on October 29, 2024, 07:13:29 am

This qualifier informs the compiler that within the entire program there is no code that will change the value of the parameter while the procedure/function is executing.

That is so unbeliveable I had to test it. And it turns out to NOT be true. A variable CAN be modified while some other rountine is "using" the variable as a constref.

Fibonacci, you need to understand what has happened here, you have, by using constref, promised no other process will mess with that variable. That, as far as the compiler is concerned, was a solemn promise, by you, to your compiler.

And you went on to break that promise ! There is a matter of trust here, your compiler will will not, now trust you. It will forever be thinking you are lying to it ! "An integer ? I bet he is going to try and sneak a double in there when I am not watching". The compiler will be on edge, never comfortable, never trusting you again.

I suggest you uninstall the compiler and reinstall ! Its the only way forward.

Davo

Title: Re: Strange bugs with Advanced Records - SOLVED
Post by: Martin_fr on October 29, 2024, 09:37:55 am

Quote

Code: Pascal [Select][+][-]
V:= U;

The record contains a dyn-array. Which is a pointer. The above statement copies that pointer.

V.I and U.I are now the same array (with a ref count of 2).

Title: Re: Strange bugs with Advanced Records - SOLVED
Post by: Fibonacci on October 29, 2024, 09:47:40 am

@dbannon :D

Quote from: Martin_fr on October 29, 2024, 09:37:55 am

Quote
Code: Pascal [Select][+][-]
V:= U;

The record contains a dyn-array. Which is a pointer. The above statement copies that pointer.

V.I and U.I are now the same array (with a ref count of 2).

Code: Pascal [Select][+]

  writeln('@V = @U? ', pointer(V) = pointer(U));
  V:= U;
  writeln('@V = @U? ', pointer(V) = pointer(U));

Code: Text [Select][+]

@V = @U? FALSE
@V = @U? TRUE

He is right.

Title: Re: Strange bugs with Advanced Records
Post by: Bogen85 on October 29, 2024, 10:12:17 am

Quote from: Joanna from IRC on October 29, 2024, 06:33:55 am

I thought it might be for multi threaded programs because of
Quote
This qualifier informs the compiler that within the entire program there is no code that will change the value of the parameter while the procedure/function is executing.
This means that not only the parameter, but also the variable (in above example: x) passed by the caller (e.g. a global var) must not be changed until the call with the constref parameter has returned.

But why only for multi threaded programs? Why not just in general?

Many find it useful to limit the side effects and state mutations when calling functions, and want to make any side effects or state mutations obvious and explicit.

If the compiler can help us with that, that is very useful. And 440bx already explained it well.

Title: Re: Strange bugs with Advanced Records
Post by: Martin_fr on October 29, 2024, 10:58:27 am

Quote from: Bogen85 on October 29, 2024, 10:12:17 am

If the compiler can help us with that, that is very useful. And 440bx already explained it well.

You may perceive it as "help by the compiler" => but it is not.

Code: Pascal [Select][+]

...(const foo: TFooType)

Is you helping the compiler (helping to optimize). Because you are telling the compiler: "foo will not change. Go ahead optimize".

And that means not to change by any means.

Code: Pascal [Select][+]

procedure somethnig(const foo: integer);
begin
  writeln(foo);
  CallSub;
  inc(foo);
  writeln(foo):
end;

If you call the above code with "x=1", and if "CallSub has a pointer to "x", and changes that value to -1 => then the above code has no defined behaviour.

If not optimized (as currently, since the compiler is not yet that clever) then it would print: 1 then 0 (after inc(-1))

But if optimized it would print: 1 then 2
Because the compiler would be allowed to keep a copy of the value 1 somewhere were CallSub could not reach it, and use that for optimization.

And this is exactly according to the documentation of const parameters.
And as you can see nothing to do with threads.

And btw, you do that with refcounted data (ansistring, dyn array) and you can get wrong data and crashes (with any current fpc version)

The fact that the compiler may give you an error if you try "foo := ... => is just that the compiler caught you lying. But, if you disguise that lie well enough, the compiler wont spot it. And then the result may be a program that does not behave (or may stop behaving in some future version of fpc, when the compiler gets better at optimizing)

Ok in this example "x" would be passed by value, so Callsub would need a pointer to foo.

Title: Re: Strange bugs with Advanced Records
Post by: Bogen85 on October 29, 2024, 12:16:56 pm

Quote from: Martin_fr on October 29, 2024, 10:58:27 am

You may perceive it as "help by the compiler" => but it is not.
...
Is you helping the compiler (helping to optimize). Because you are telling the compiler: "foo will not change. Go ahead optimize".

And that means not to change by any means.

I still take that as a win. (and a mutual helping of each other out). :D

I take the approach that I by default I expect the items I pass to functions, that the function will not change the caller's view of those items (or that the function will modify what was passed in, even if it just it's own copy of those), unless I explicit want such a change to take place (or that the parameters are explicitly free to use for whatever local variables because of no const/constref). So const and constref helps clarify those assumptions.

Though I agree, how the compiler actually does it, might not be completely aligned with that, but so far those assumptions have not bitten me.

Title: Re: Strange bugs with Advanced Records
Post by: Warfley on October 29, 2024, 01:31:09 pm

Quote from: ad1mt on October 29, 2024, 09:30:52 am

After calling the TrashREC procedure, the first array element of U is also incorrectly overwritten, as follows:

U=99 99 99 99 99 V=99 99 99 99 99 TrashREC U=100 99 99 99 99 V=100 99 99 99 99
While investigating this, I found this page in the wiki...
https://wiki.freepascal.org/management_operators (https://wiki.freepascal.org/management_operators)
This gives an explanation of what is happening, and a solution to the problem. The fixed/working program is:

The reason you have this problem is quite simply is that dynamic arrays are (unlinke dynamic strings) not copy on write. Meaning if you have multiple references to the same array, it will not be made unique on access.
For strings there is the function UniqueString (https://www.freepascal.org/docs-html/rtl/system/uniquestring.html), but afaik nothing comparable exists for arrays yet.

For your purposes the most simple way of doing the deep copy would be to simply use setlength, as it ensures a refcount of 1, meaning it can be used to create a unique copy:

Code: Pascal [Select][+]

class operator REC.copy(constref v1:REC; var V2:REC);
begin
  v2.I:=v1.I;
  SetLength(v2.I, Length(v2.I));
end;

Personally I think a deepcopy intrinisc utilizing RTTI would be a very useful addition to the compiler. All thats needed to implement it is already part of the InitRTTI table used for managed types anyway

Title: Re: Strange bugs with Advanced Records - SOLVED
Post by: Joanna on October 29, 2024, 01:37:21 pm

I read the constref wiki page French {using translator} And it’s quite a different story. It said it’s for interfacing with External code in other languages ?

Quote

Les notes de version de la version 2.6 suggère que vous ne devriez utiliser ceci que pour l'interface avec des routines externes écrites dans d'autres langages, où ce type de passage de paramètre est requis.

I did sound a bit far fetched that passing a value as a constref could stop it being changed in any other part of the program. :D

The only way I can think of to do something like that is to use a property with setter with Boolean variable inside to temporarily lock it so it can’t be changed...

Title: Re: Strange bugs with Advanced Records - SOLVED
Post by: Warfley on October 29, 2024, 02:57:41 pm

The Wiki, neither the french nor english pages, are documentation. They are use provided content. The freepascal website is down right now, so I cannot look up the actual documentation, but if I remember correctly the idea behind constref is that in some situations you must ensure that a variable is passed as reference. Because "const" always tries to do the most efficient way of passing (as for types that fit in registers, it's less efficient to pass by reference than by register), it cannot be guaranteed that it is passed by reference. Var which has this guarantee on the other hand makes optimizations more difficult, because if the variable can or will be changed, some optilizations like changing order of execution, may impact the result.

So constref is a pass by reference, where you as a programmer promise the compiler that you won't change it. It's somewhat semantically enforced, but not fully, e.g.

Code: Pascal [Select][+]

procedure Foo(constref A: Integer);
begin
  PInteger(@A)^ := 42;
end;

Because it's constref you can take the pointer of it (which may be required for interfacing C style functions), but because the FPCs typesystem has (unlike e.g. C's type system) no concept of constness outside of parameters, the constness is gone.

The reason why you need constref for the copy operator is very simple. Imagine the parameter would be passed by value, then it would needed to be addref'd or copied first, which would interfere with the implementation of the copy operator. So in order to ensure that it will never be copied when calling copy, you enforce a reference by using constref

Title: Re: Strange bugs with Advanced Records - SOLVED
Post by: Thaddy on October 29, 2024, 05:55:47 pm

That is my hack, but here that is not the case: S has outer scope, so priority scope over T.
That is not only expected, but correct, since the contract between the programmer and the compiler is not touching T within the scope. It is also simply bad programming.

Title: Re: Strange bugs with Advanced Records - SOLVED
Post by: 440bx on October 29, 2024, 07:40:33 pm

I see there is a lot of misunderstanding as to what "constref" means and does...

Consider the following code:

Code: Pascal [Select][+]

type
  TSOMETYPE = record
    Field1    : integer;
    Field2    : integer;
    Field3    : integer;
    Field4    : integer;
  end;
 
var
  MyVar : TSOMETYPE;
 
function TheFunction(constref TheParameter : TSOMETYPE; var ANumber : integer) : boolean;
var
  LocalVar : integer;
begin
  LocalVar := TheParameter.Field3;
 
  ANumber := 5;  { any value will do }
 
  result := TRUE;
end;
 
begin
  { note that "MyVar" being "constref" does NOT prevent passing }
  { MyVar.Field3 as a "var" parameter that IS modified.         }
 
  { "constref" only determines what can be done in the scope in }
  { which it is active and does NOT prevent modifying the record}
  { or its fields by other means and, that's LEGAL.             }
 
  TheFunction(MyVar; MyVar.Field3); 
end.
 

Note that "MyVar" is constref, yet the field "MyVar.Field3" can be modified by the function because that field is passed by ~~value~~ reference. It may seem strange but, that's perfectly legal (not to mention very useful.)

HTH.

ETA:

Corrected "value" to "reference" (thank you Martin)

Title: Re: Strange bugs with Advanced Records - SOLVED
Post by: Martin_fr on October 29, 2024, 07:53:39 pm

"var ANumber" => that is passed by reference.

So this is actually breaking the const rule. The compiler just can't detect it. And currently no version of fpc does any optimization (that I know of) that would lead to an error as result of this violation. However, a future version may have such optimizations.

"constref" is described on this page. And - as far as I know - falls under the same conditions
https://www.freepascal.org/docs-html/ref/refsu67.html

Quote

It is the programmer who tells the compiler that the contents of the const parameter will not be changed when the routine is executed

And that does NOT say "by code inside the routine directly accessing the field".

Title: Re: Strange bugs with Advanced Records - SOLVED
Post by: Warfley on October 29, 2024, 08:46:56 pm

Quote from: Martin_fr on October 29, 2024, 07:53:39 pm

And currently no version of fpc does any optimization (that I know of) that would lead to an error as result of this violation. However, a future version may have such optimizations.

A bit curious, is the constness information compiled into llvm IR when the llvm backend is used? If so, it might be relevant because llvm has such optimizations that rely on constness.
To be fair my experience with llvm dates back to llvm 3.8 and I never used FPC with llvm (even though it's definitely on my list to try out)

Title: Re: Strange bugs with Advanced Records - SOLVED
Post by: 440bx on October 29, 2024, 09:38:43 pm

Quote from: Martin_fr on October 29, 2024, 07:53:39 pm

So this is actually breaking the const rule.

I've come to interpret the "const rule" a little bit differently than what the FPC documentation states.

For me, the promise is that the parameter or, in the case of a record, parameter fields will _not_ be modified using that parameter. IOW, it does not promise the parameter isn't changed using an alias (which is what the "var" parameter in the example I provided does.)

I believe that when an alias is used, as in the example I gave then, it is entirely the programmer's responsibility to ensure no undesirable side effects occur no matter what the compiler decides to do (as long as what the compiler does is correct.)

Some compilers have optimization levels that explicitly preclude the use of pointer aliasing and offer ways for the programmer to declare that a specific sequence of code uses pointer aliasing to inform the compiler some optimizations cannot be applied to that sequence of code.

The example I provided in my previous post is a bit too simplified. Consider the record as being made of other records (instead of just integers.) It is not uncommon for code to need read access to many of the records and read-write to only one or two. That's when passing the entire record as "constref" and one or two of its component records as "var" makes sense and is very useful.

Title: Re: Strange bugs with Advanced Records - SOLVED
Post by: Martin_fr on October 29, 2024, 10:15:37 pm

Run this code

Code: Pascal [Select][+]

program Project1;
 
uses SysUtils;
 
//procedure Foo(constref s1: Ansistring; var s2: Ansistring);
procedure Foo(const s1: Ansistring; var s2: Ansistring);
var
  m: Pointer;
begin
  writeln(s1);
  s2 := StringOfChar('x', 100+ Random(99));
  m := AllocMem(10);
  FillChar(m^, 10 , 1);
  writeln(Length(s1), s1);
  writeln(Length(s2), s2);
end;
 
 
var a,b: Ansistring;
begin
  a := 'abc ' + IntToStr(Random(99));
  b := 'abc ' + IntToStr(Random(99));
 
  Foo(a, a);
  readln;
end.
 

It is essentially the same you have, just with ansistring.

And in more complex apps, with more memory used and freed over time, you can get crashes and all.

The constref behaves a bit more forgiving.

This is because in todays FPC I don't know any optimization that would allow me to exploit (in a negative way) the broken promise. But a future FPC may bring such an option.

Mind: "I don't know any" doesn't mean that there isn't.

Title: Re: Strange bugs with Advanced Records - SOLVED
Post by: 440bx on October 30, 2024, 01:06:42 am

That's why I said that it is ultimately the programmer's responsibility to ensure there are no undesirable side effects.

For instance, the compiler does not know if some area of memory being referenced in some function/procedure is also being referenced by a different function/procedure in another thread. It is the programmer who must know and account for situations like that.

Title: Re: Strange bugs with Advanced Records - SOLVED
Post by: Joanna on October 30, 2024, 01:26:15 am

Is there any way to prevent the fields of a record Or object passed as a constant from being modified?

Title: Re: Strange bugs with Advanced Records - SOLVED
Post by: 440bx on October 30, 2024, 04:10:18 am

Quote from: Joanna from IRC on October 30, 2024, 01:26:15 am

Is there any way to prevent the fields of a record Or object passed as a constant from being modified?

Depends...

if what you're asking for is some way to tell the compiler to prevent modification in all circumstances then the answer is _no_ because there are plenty of dirty tricks that can be used to sidestep the compiler's guard rails.

That said, if your data is structured the right way then the answer can be _yes_. Here is an example, say you have a variable of type record TMYRECORD and you want to make certain that it cannot be changed by any other code then, what you do is use the O/S facility VirtualProtect to change the memory attributes to read-only. Important to know that to do that the variable _must_ have been allocated using VirtualAlloc, otherwise using VirtualProtect on it can cause all kinds of problems. For something like this to work well, upfront design is most definitely necessary but, well worth it.

The bottom line is always the same: the programmer has to know what he/she is doing. The compiler can help catch mistakes but cannot catch all mistakes. This has always been true but, it is particularly true in multi-tasking systems.

Title: Re: Strange bugs with Advanced Records - SOLVED
Post by: Thaddy on October 30, 2024, 07:18:54 am

Quote from: Joanna from IRC on October 30, 2024, 01:26:15 am

Is there any way to prevent the fields of a record Or object passed as a constant from being modified?

If the record is compiled in {$J-} state, you can not modify it. (It usually ends up in .rodata, read-only data.)
{$J+/-} is a local directive, so you can use {$push}{$J-}record declaration{$pop}. Only in {$J+} state the record is writable and that is the default.
Small demo:

Code: Pascal [Select][+]

{$mode objfpc}
type 
  trec1 = record
  a:integer;
  end;
     
{$push}{$J-}
const
  rec1:trec1 = (a:100);
{$pop}
begin
  writeln(rec1.a);
  { in J- state this will not compile
    in J+ state it will compile }
  rec1.a:= 101;
  writeln(rec1.a);
end.

This is applicable to any typed const, not only records, everything.
You can even pass it as a const parameter to a procedure, does not matter, you can't modify it at all anyway.
But in J+ state, even if you pass it as a const parameter, depending on scope there are ways to modify it if you are not careful. See the example in the documentation for const (the one with T and S mentioned above)

But in the above example the record is not modifiable anywhere.
To show that effect:

Code: Pascal [Select][+]

{$mode objfpc}
type 
  trec1 = record
  a:integer;
  end;
     
{$push}{$J+} // + state
const
  rec1:trec1 = (a:100);
{$pop}
procedure dosomething(const value:trec1);
begin
  writeln(value.a);
  // value.a :=104;     // this will not compile, but...
  rec1.a := 102;        // rec1 has outer scope
  writeln(value.a);     // so this ends up modified too..but only in J+ state.
end;
 
begin
  writeln(rec1.a);
  { in J- state this will not compile
    in J+ state it will compile }
  rec1.a:= 101;
  writeln(rec1.a);
  dosomething(rec1);
end.

Play with the +/- switch.

Title: Re: Strange bugs with Advanced Records - SOLVED
Post by: Warfley on October 30, 2024, 12:23:35 pm

Quote from: 440bx on October 30, 2024, 01:06:42 am

That's why I said that it is ultimately the programmer's responsibility to ensure there are no undesirable side effects.

For instance, the compiler does not know if some area of memory being referenced in some function/procedure is also being referenced by a different function/procedure in another thread. It is the programmer who must know and account for situations like that.

Yes and no. It's the responsibility of the programmer to follow the defined semantics of the language. And in Pascal the defined semantic for a const(ref) parameter is that it will not change during the rumtime, so you as a programmer must make sure that this doesn't happen. It's a two way contract so to speak. You promis to make sure the parameter doesn't change, and in exchange the compiler promises that it generates correct code.

For example, if I compile the following code for 32 and 64 bit I get differing results:

Code: Pascal [Select][+]

type
  TMyRec = record
    A: Integer;
  end;
 
procedure Foo(const r: TMyRec; var A: Integer);
begin
  A:=42;
  WriteLn(r.A);
end;
 
var
  r: TMyRec;
begin
  r.A:=32;
  Foo(r, r.A);
end.

The reason is that in one case the parameter is passed by register in the other it's passed by reference. This means this code has undefined semantics, which only happens because I broke the promise I made the compiler through passing a "const".

There are just things that you as a programmer must never make assumptions about. It's called undefined behavior and no matter how good you know the language and the compiler, you should never think you know whats happening there.

I had a great experience with C++, because prior to C++20 bit representation (and thereby overflow) for the int types was not defined. So I had a for loop like this:

Code: C [Select][+]

for (;myvar>0;++myvar) {
  ...
}

Which should loop until overflow happend, and the compiler just turned it into:

Code: C [Select][+]

while (true) {
  ...
}

Because overflow is not defined, so the compiler reasoned, a value >0 that will only ever be incremented will never get smaller and therefore this condition can never be met. Mathematically a sound argumentation, but of course vary nasty to figure out in my situation.

Long story short, you as a programmer only ever are responsible to follow the semantics of the language. If something falls outside of the defined behavior of the language, you should not make any assumptions about it

Title: Re: Strange bugs with Advanced Records - SOLVED
Post by: Joanna on October 30, 2024, 01:39:22 pm

Interesting tricks but still it’s kind of misleading declaring a record Parameterconst if it in fact is not. :D

Title: Re: Strange bugs with Advanced Records - SOLVED
Post by: Thaddy on October 30, 2024, 02:09:18 pm

It is const within the scope. Examples that show you otherwise is imo bad programming and that includes my own second example above. That is merely an illustration, as is the anti-pattern in the documentation, which is meant to be an ...anti-pattern.

@Warfley forgot another one: the calling convention also impacts the behavior of how const parameters are passed.

Title: Re: Strange bugs with Advanced Records - SOLVED
Post by: Warfley on October 30, 2024, 04:11:56 pm

Quote from: Joanna from IRC on October 30, 2024, 01:39:22 pm

Interesting tricks but still it’s kind of misleading declaring a record Parameterconst if it in fact is not. :D

This is the asymmetry of the definition. By declaring a parameter const you promise that your function will not change that parameter (mostly enforced by the compiler). But thats just one side of the equation, the other side is that the caller must also respect that contract. If the function is declared with one const parameter and one var parameter, i.e. one parameter where my function promises that it will not be changed, and one where my function says it will change it, then passing the same parameter in both, "tricks" the function into violating it's own contract. So both parties must agree to not change the variable.
And the thing is, the compiler can only enforce the former, not the latter (well you could also enforce the latter for most cases, by doing a bit of data flow analysis, but I guess thats not yet implemented in -Oodfa).

Note that the function may be provided externally, e.g. I write a lot of libraries, so I provide a lot of functions where I have absolutely no idea who's going to use them in which context. On the other hand if you use a library, you have no influence on the functions you are calling. So I as a library writer can put const everywhere where I don't intend to change the value of the parameter, but thats only half the story. The user of that library also must respect that "contract" when using the function

Title: Re: Strange bugs with Advanced Records - SOLVED
Post by: 440bx on October 30, 2024, 05:05:22 pm

Quote from: Warfley on October 30, 2024, 12:23:35 pm

Long story short, you as a programmer only ever are responsible to follow the semantics of the language.

For that part, I qualify that with "most of the time" or "if convenient".

Quote from: Warfley on October 30, 2024, 12:23:35 pm

If something falls outside of the defined behavior of the language, you should not make any assumptions about it

I definitely don't make any assumptions. I make sure the compiler behaves as I expect it to and, not only that, as it _has_ to.

That's the reason I only use "const" when dealing with ordinal types and I am fully aware that some types are ordinal in 32 64 bit but not in 64 32 bit, e.g, qword. Anything I haven't done in the past, I make it a point to look at the generated assembler to ensure it is the way I expect it to be (and the way the compiler should be coding it.)

ETA:

I should have noted that I normally place a comment pointing out that part of a "constref" record is also being passed by reference as a different parameter. Just to eliminate any "surprise" factor.

Title: Re: Strange bugs with Advanced Records - SOLVED
Post by: Thaddy on October 30, 2024, 05:11:24 pm

The only thing what we really should be discussing if fpc needs a true static variable: as it is whe have the choice beween typed const in J+ state OR J- state. We should have both, without switches. And it confuses people because of the unfortunate naming.

Title: Re: Strange bugs with Advanced Records - SOLVED
Post by: Warfley on October 30, 2024, 07:42:16 pm

Quote from: 440bx on October 30, 2024, 05:05:22 pm

I definitely don't make any assumptions. I make sure the compiler behaves as I expect it to and, not only that, as it _has_ to.

That's the reason I only use "const" when dealing with ordinal types and I am fully aware that some types are ordinal in 32 64 bit but not in 64 32 bit, e.g, qword. Anything I haven't done in the past, I make it a point to look at the generated assembler to ensure it is the way I expect it to be (and the way the compiler should be coding it.)

I whole heartedly disagree with that approach. When using a high level language you should not make any assumptions (or expectations) about the underlying assembly. To give an example in C the "char" type is the only ordinal type which is neither signed nor unsigned. The reason for this is simply, some processors are faster with signed chars, others with unsigned. So in order to produce the optimal code for any CPU, C does not make any assumptions about that.
I noted earlier that C++ had up until C++20 no defined representation of signed integer types. The reason for that is, coming from C, it may be implemented on machines that use Sign and Magnitude, 1s complement or 2s complement. This has only been changed recently because 2s complement is so common that it doesn't make sense to optimize for the others anymore.

But what I'm getting at here is, if you write a program in valid C, or C++, or Pascal, or any other high level language, it should work the exact same on any machine and any cpu. No matter if it's a 64 bit little endian x64 CPU, or a 16 bit big endian motorola 6809 chip.
In C it goes even so far that things like converting bit representations (through pointers or unions) are actually not allowed by the standard, meaning if you write fully standard C without any implementation defined or undefined behavior, it will run exactly the same on any cpu.

So whenever writing code in a high level language it's best to assume it's implemented using fairy dust and magic, and don't think about what happens on an assembler level. Assumptions about the generated assembler works ok-ish with a language like Pascal which does not have much undefined behavior and frankly has rather little optimizations, but you still shouldn't bet on it, as there is still constant work on the FPC, and as I said previously, I'm personally very curious about the LLVM backend, as LLVM can do some crazy optimizations.

PS: also with Pascal or C I'm of course talking about rather low level languages, where there is an "obvious" way on which assembly they result in. If you go to much higher level languages such as Haskell or something similar, thinking about your code in assembly is going to give you much more trouble

Title: Re: Strange bugs with Advanced Records - SOLVED
Post by: 440bx on October 30, 2024, 09:44:25 pm

Quote from: Warfley on October 30, 2024, 07:42:16 pm

I whole heartedly disagree with that approach.

I have no doubt you have plenty of company there.

Quote from: Warfley on October 30, 2024, 07:42:16 pm

When using a high level language you should not make any assumptions (or expectations) about the underlying assembly.

I will say it again: I don't make assumptions and I verify that the compiler did what I expected it to do.

Compilers don't do magic, they follow rules to generate code, that's true of optimizers too, they follow rules. Compilers, unlike programmers cannot break the rules because if they do, that's reported as a bug. If a compiler breaks a rule I need it to enforce, I'll change the code to have it generate the code I want. IOW, the compiler is tool that works for _me_, I don't work for the compiler.

When a programmer breaks the rules (which I sometimes do) then it is on his/her shoulder to ensure breaking whatever rule got broken does not have undesirable consequences, if it does fix it and, if it doesn't enjoy :)

One rule that should ideally never be broken is, someone who doesn't know what they're doing shouldn't be breaking rules (unless they want to learn the hard way.)

Title: Re: Strange bugs with Advanced Records - SOLVED
Post by: Warfley on October 31, 2024, 05:59:16 am

Quote

I don't make assumptions and I verify that the compiler did what I expected it to do.

Assumptions that you verify are still assumptions. Note that when I googled "expectation synonym" the very first result was "assumption" :)

Quote from: 440bx on October 30, 2024, 09:44:25 pm

Compilers don't do magic, they follow rules to generate code, that's true of optimizers too, they follow rules. Compilers, unlike programmers cannot break the rules because if they do, that's reported as a bug. If a compiler breaks a rule I need it to enforce, I'll change the code to have it generate the code I want. IOW, the compiler is tool that works for _me_, I don't work for the compiler.

But these rules can change with future versions of the Compiler, or when used on different targets. When I write code in Pascal it should work independently of which configuration of Compiler and target platform is used. I neither can nor want to be bothered every time my code is used in a different configuration to verify it does whatever I originally intended it to do.

If not it would be completely impossible to write libraries in any way shape or form, because when I write a library, there is absolutely no way I can know how it will be used in the end, and I need to write it in a way that it works in pretty much all configurations now and in the future. And this is possible because programming languages define their semantics independently of their target system or configuration. A valid standard conforming c program written in 1980s for a 8080, should still work correctly today using modern GCC on a x64.

Title: Re: Strange bugs with Advanced Records - SOLVED
Post by: 440bx on October 31, 2024, 07:06:28 am

Quote from: Warfley on October 31, 2024, 05:59:16 am

Assumptions that you verify are still assumptions. Note that when I googled "expectation synonym" the very first result was "assumption" :)

Personally, in my book, once something is verified it is no longer an assumption. It's kinda like that by definition.

Quote from: Warfley on October 31, 2024, 05:59:16 am

But these rules can change with future versions of the Compiler, or when used on different targets.

That's true. In those cases the assumption(s), if any, need to be verified again to account for the changes that took place.

I am not suggesting that rules should be broken in a cavalier manner. What I am "suggesting" is that there are rules the compiler _wishes_ it could enforce but cannot. Breaking those rules is very low risk (as long as the programmer knows what he/she is doing.) Among those, passing a large structure as a constant by reference and passing some other records that are part of it by reference (and not constant.) The important thing when that is done is to always reference those other records using their reference (var) identifier because, if referenced using the constant identifier then the compiler could mistakenly (and correctly) presume the value has not changed. Because of that, values which are not constant should be referenced using the "var" identifier. That complies with the compiler's rules because the compiler cannot presume any field in those records are constant because they are passed by reference and, the compiler _knows_ that therefore it cannot indulge in optimizations that only work when the values are constant.

Again, if the programmer knows what he/she is doing, things will work fine.

A bit ironically, what I described follows the rules and, depending on how things are interpreted, also keeps the promise(s) made to the compiler, i.e, the values are never changed using the constref identifier and values that are changed are changed using the "var" identifier, therefore the compiler is being kept informed of what is happening using the methods the compiler expects to be kept informed.

As long as it's done the right way, there is no problem there.

Title: Re: Strange bugs with Advanced Records
Post by: ad1mt on November 01, 2024, 09:04:34 pm

Quote from: Warfley on October 29, 2024, 01:31:09 pm

The reason you have this problem is quite simply is that dynamic arrays are (unlinke dynamic strings) not copy on write. Meaning if you have multiple references to the same array, it will not be made unique on access.
For strings there is the function UniqueString (https://www.freepascal.org/docs-html/rtl/system/uniquestring.html), but afaik nothing comparable exists for arrays yet.

For your purposes the most simple way of doing the deep copy would be to simply use setlength, as it ensures a refcount of 1, meaning it can be used to create a unique copy:
Code: Pascal [Select][+][-]
class operator REC.copy(constref v1:REC; var V2:REC);
begin
v2.I:=v1.I;
SetLength(v2.I, Length(v2.I));
end;

Personally I think a deepcopy intrinisc utilizing RTTI would be a very useful addition to the compiler. All thats needed to implement it is already part of the InitRTTI table used for managed types anyway

I find all this very disappointing.

I've programmed in low-level languages like assembly and C, so I know about all these complexities.
But my view is that a high-level language should hide the low-level details so that you don't have to worry about them, and the code does what you would expect. Programmers should not have know the deep internals of how data types are represented and copied to make their programs work.

If the programmer wants to gain the efficiency of copy-on-write and having the internals of records be pointers to external data objects, then I think that those features should be enabled by a switch, and with the switch having a warning that hidden dangers are present. Then expert programmers, who understand the implications, can turn the features on if they wish.

I had a problem several months ago with FPU exceptions on Intel 32bit CPU's. I turns out that FPU exceptions do not work correctly on Intel 32bit CPU's, unless a special compiler switch is turned-on. I argued that programmers should not have to turn on an obscure switch to make their program work correctly. The correct behaviour should be enabled by default and if the programmer wants to gain a small run-time efficiency by having unsafe FPU exceptions code, then it should be enabled with a switch (rather than the other way round). The existing compiler behaviour means that code using FPU exceptions is broken by default on 32bit Intel CPU's.

Title: Re: Strange bugs with Advanced Records
Post by: Warfley on November 01, 2024, 10:07:57 pm

Quote from: ad1mt on November 01, 2024, 09:04:34 pm

I find all this very disappointing.

I've programmed in low-level languages like assembly and C, so I know about all these complexities.
But my view is that a high-level language should hide the low-level details so that you don't have to worry about them, and the code does what you would expect. Programmers should not have know the deep internals of how data types are represented and copied to make their programs work.

If the programmer wants to gain the efficiency of copy-on-write and having the internals of records be pointers to external data objects, then I think that those features should be enabled by a switch, and with the switch having a warning that hidden dangers are present. Then expert programmers, who understand the implications, can turn the features on if they wish.

But I mean the problem was quite the opposite, the problem is not that arrays do hidden copy on write, but that they specifically do not do that. An array is just a dumb pointer, that is copied around when you copy the record.
The problem here is specifically that the fpc does not try to be smart and does not add copy-on-write (as it for example does with dynamic strings). If arrays had copy on write like strings have, your problem wouldn't exist but instead the issue is very specifically is that FPC just does a dumb pointer copy and nothing more.

You could create a deepcopy function for simple records using RTTI. The problem of course arises when you have records where data is coupled to semantics, e.g. if you have a record that "owns" a pointer, just copying that pointer will not work. Meanwhile when you have a record that has a shared pointer, deepcopying that pointer means that no one is responsible for the deep copy.

Title: Re: Strange bugs with Advanced Records
Post by: ad1mt on November 02, 2024, 09:08:10 am

Quote from: Warfley on November 01, 2024, 10:07:57 pm

An array is just a dumb pointer, that is copied around when you copy the record.

My point is this...

I did not understand the internals of how the compiler dealt with a record containing a dynamic array. Then when I wrote the code v1:= v2; procl(v1,v2); expecting the behaviour to be the same as any other type, it did not work. The proc_call overwote v1 as well as v2, even though in the definition of proc, v1 was specified as const.

This means that the behaviour of the code was breaking already established rules about the expected behaviour of const parameters in procedure calls, and is inconsistent.

The implications these kinds of complexities, means that no-one can safely code in Pascal until they are an expert that knows everything, right down to all the obscure details of how the compiler works. I.M.O. this defeats the whole philosophy of Pascal as being a safe language.

Title: Re: Strange bugs with Advanced Records
Post by: 440bx on November 02, 2024, 09:24:47 am

Quote from: ad1mt on November 02, 2024, 09:08:10 am

The implications these kinds of complexities, means that no-one can safely code in Pascal until they are an expert that knows everything, right down to all the obscure details of how the compiler works. I.M.O. this defeats the whole philosophy of Pascal as being a safe language.

No compiler can compensate for an inadequate level of knowledge in a programmer.

It is unfortunate that programmers are commonly mislead/encouraged to believe the compiler can protect them from themselves. There are plenty of ways to trip a compiler, any compiler. The reason is simple, the compiler only has compile time information, the guard rails it enforces are usually very easy to circumvent at runtime and, that, does _not_ reflect badly on the compiler, it's reflects on the programmer. The programmer has to know what he/she is doing and cannot blame the compiler for not always catching code that won't work properly.

Title: Re: Strange bugs with Advanced Records
Post by: Warfley on November 02, 2024, 01:37:10 pm

Quote from: ad1mt on November 02, 2024, 09:08:10 am

I did not understand the internals of how the compiler dealt with a record containing a dynamic array. Then when I wrote the code v1:= v2; procl(v1,v2); expecting the behaviour to be the same as any other type, it did not work. The proc_call overwote v1 as well as v2, even though in the definition of proc, v1 was specified as const.

That is not special knowledge, if you know how records work and you know how dynamic arrays work, it's exactly what you would expect. If you don't know either, you should read into that before using those types.

Basically a record is a simple composite type, where an assignment is equivalent to assigning all of the fields:

Code: Pascal [Select][+]

rec1 := rec2;
// Is basically the same as
rec1.Field1 := rec2.Field1;
rec1.Field2 := rec2.Field2;
rec1.Field3 := rec2.Field3;
...

It's a very dumb copy.

Dynamic arrays are reference counted pointers. An assignment of a dynamic array does not copy the array but just the pointer to the array:

Code: Pascal [Select][+]

arr1 := arr2;
// Is the same as
Pointer(arr1) := @arr2[0]; // Copy pointer
IncrementRefcount(arr2);

So by putting these two facts together, a record which contains a dynamic array will copy all the fields of the record as if they are assigned, meaning because on assignment of an array the the data of the array is not copied, it also won't be when assigning a record.
In technical terms, it does a shallow copy.

There is absolutely nothing unexpected about the combination of those two, if you are familiar with each one individually

Title: Re: Strange bugs with Advanced Records - SOLVED
Post by: Thaddy on November 02, 2024, 02:05:15 pm

I have shown that many times, including in the wiki, but here again:

Code: Pascal [Select][+]

program reccopy;{$mode objfpc}{$modeswitch advancedrecords}
type 
  Tmyrec = record
    a,b,c:integer;
    class operator copy (constref value:Tmyrec; var result:Tmyrec);
  end;
// in Delphi this one is called assign
class operator Tmyrec.copy (constref value:Tmyrec; var result:Tmyrec);
begin
  // move (sizeof) takes only the fields, not the methods.
  move (value,result, sizeof(tmyrec));
  writeln('copy called');// <--- just to make it very clear...
end;
var
  x,y :Tmyrec;
begin
  x.a := 100;
  x.b := 200;
  x.c := 300;
  y:=x; //invokes copy
  y.a := 500;// change a value, see what happens..
  writeln (x.a:4, x.b:4, X.c:4);
  writeln (y.a:4, y.b:4, y.c:4);  
end.

You use the copy operator to make a deep copy instead of a shallow - pointer- copy on assignment.
I am almost sure ALL of this has been shown in this thread already.
Management operators are never called directly, they are hidden from view.

Lazarus

Free Pascal => General => Topic started by: ad1mt on October 28, 2024, 08:15:52 pm