Lazarus

Programming => General => Topic started by: jamie on August 18, 2019, 03:07:51 am

Title: I need clarification with "open Arrays"
Post by: jamie on August 18, 2019, 03:07:51 am
This for example;
Code: Pascal  [Select]
  1. function SearchTable(CodePageArr: array of word; id: cardinal): word;  
  2.  

if the array is very large, is this getting pushed on the stack each time it gets called?

Shouldn't it be a "VAR" reference in there?
 
Title: Re: I need clarification with "open Arrays"
Post by: winni on August 18, 2019, 03:18:56 am
No. Open arrays are only a pointer. The logic with the VAR parameters is broken since Delphi1 - in the pascal logic of N. Wirth.

Winni
Title: Re: I need clarification with "open Arrays"
Post by: jamie on August 18, 2019, 03:32:46 am
That's strange because reading my Delphi  help file, one of my old tools (3) it states that it does follow the VAR. VALUE or CONST and puts this on the stack  I just found a reference on the net for FPC and it also states the same as Delphi.
 
 A VALUE which is what I showed here will make a copy to the stack so that you can make changes to it locally but not effect it globally and so on with the rest.

 Looking at some code in various units that are used in Lazarus and fpc I have been finding questioned code practices..

 Thanks for the heads up, I am going to run a debug session and view some ASSEMBLER code for varication  :)
Title: Re: I need clarification with "open Arrays"
Post by: jamie on August 18, 2019, 04:15:57 am
I just ran some test and it does indeed make copies of the array if you don't use the "VAR" pass by Ref.

 So I guess it does follow along.. I am trying to speed up some code and looking around I just happen to see that in a file. Actually I see that used without "Var" in many places.

 I also found out that Fpc 3.0.4 does not support functions that accept Open Arrays to be inlined.
 Not sure if this has been rectified in later versions yet. its not a big deal because it would only help in some cases.
Title: Re: I need clarification with "open Arrays"
Post by: 440bx on August 18, 2019, 04:39:53 am
I was going to give an example showing that it  does make a copy of the array (unless you specify "var") and in the process, ran into a bug related to the debugger. 

The following small program, if compiled for 64bit and a breakpoint is placed on the program's "begin" (line 25) then an illegal instruction exception occurs.

The problem does not happen in 32bit.

Code: Pascal  [Select]
  1. program _Arrayof;
  2.  
  3. {$LONGSTRINGS ON}
  4.  
  5.  
  6. type
  7.   T500 = array[0..500] of DWORD;
  8.  
  9. var
  10.   A     : T500;
  11.  
  12. procedure InitArray(p : array of DWORD);
  13. var
  14.   i : integer;
  15. begin
  16.   for i := low(p) to high(p) do
  17.   begin
  18.     p[i] := 7;    // any value
  19.   end;
  20. end;
  21.  
  22. var
  23.   i : integer;
  24.  
  25. begin
  26.   InitArray(A);
  27.  
  28.   for i := low(A) to high(A) do
  29.   begin
  30.     writeln('A[', i, '] = ', A[i]);
  31.   end;
  32.  
  33. end.
  34.  
Title: Re: I need clarification with "open Arrays"
Post by: WooBean on August 18, 2019, 08:01:10 am
Confirmed for FPC 3.04 Lazarus 1.8 (win 32 bit) - code executes, for FPC 3.04 Lazarus 2.04 (win 64 bit) - code fails.
More, if "procedure InitArray(p : array of DWORD);"  is declared as "procedure InitArray(var p : array of DWORD);" it works in both environments.     
My comments is that there are differences at accepting static arrays by FPC 3.0.4 as open array depending on usage of var keyword.

WooBean
Title: Re: I need clarification with "open Arrays"
Post by: ASerge on August 18, 2019, 11:52:52 am
Code: Pascal  [Select]
  1. function SearchTable(CodePageArr: array of word; id: cardinal): word;  
  2.  
if the array is very large, is this getting pushed on the stack each time it gets called?
In Delphi typically used const, which notifies the compiler that it can optimize the code and pass only the reference, otherwise it pass by value. In FPC to accurately ensure this used constref.
By the way, may be use System.IndexWord?
Title: Re: I need clarification with "open Arrays"
Post by: 440bx on August 18, 2019, 01:00:50 pm
In Delphi typically used const, which notifies the compiler that it can optimize the code and pass only the reference, otherwise it pass by value.
Just for precision's sake.  The array will always be passed by reference.  The presence of "const" tells the compiler that it doesn't need to make a copy of the array (since the values won't/cannot be changed/assigned to.)  If "const" is not specified then a copy of the array is made and all array operations are done on the copy.




Title: Re: I need clarification with "open Arrays"
Post by: LemonParty on August 18, 2019, 01:36:50 pm
Quote
Dynamic arrays are reference counted. Calling setLength(myDynamicArrayVariable, 0) virtually does myDynamicArrayVariable := nil and decreases the reference count. Only when the reference count hits zero, the memory block is released.
Rewrite InitArray as a function that return an array.
Title: Re: I need clarification with "open Arrays"
Post by: Thaddy on August 18, 2019, 01:48:03 pm
In Delphi typically used const, which notifies the compiler that it can optimize the code and pass only the reference, otherwise it pass by value.
Just for precision's sake.  The array will always be passed by reference.  The presence of "const" tells the compiler that it doesn't need to make a copy of the array (since the values won't/cannot be changed/assigned to.)  If "const" is not specified then a copy of the array is made and all array operations are done on the copy.

Only in the case of a const open array e.g:
Code: Pascal  [Select]
  1. {$mode delphi}{$H+}
  2. type
  3.   TWordArray = array of word;
  4.  
  5.   // open array
  6.   function ArrayContains(const arr:array of word;item:word):Boolean;overload;
  7.   var i:integer;
  8.   begin
  9.     Result := false;
  10.     for i :=0 to high(arr) do
  11.       if arr[i] = item then result := true;;
  12. //      if arr[i] = item then begin result := true;arr[i]:=100;exit end; // this won't work
  13.   end;
  14.  
  15.   // typed array
  16.   function ArrayContains(const arr:TWordArray;item:word):Boolean;overload;
  17.   var i:integer;
  18.   begin
  19.      Result := false;
  20.     for i :=0 to high(arr) do
  21.       if arr[i] = item then begin result := true;arr[i]:=100;exit end; // but this will allow the modification
  22.   end;
  23.  
  24. var
  25.   a:TWordArray = [1,2,3,4,5,6,7,8,9];  
  26.   b:array of word = [1,2,3,4,5,6,7,8,9];
  27. begin
  28.   writeln(arrayContains(a,3));
  29.   writeln(arrayContains(b,3));
  30. end.
So if you need to modify declare an array type.
Title: Re: I need clarification with "open Arrays"
Post by: 440bx on August 18, 2019, 02:07:31 pm
Rewrite InitArray as a function that return an array.
That's very inefficient.  The compiler will allocate stack space to hold the result of the function then copy the result into the receiving variable.



ETA:

@Thaddy

Only in the case of a const open array
No.  Even if the array is passed by value, the compiler will pass the array by reference and the reference is used to make a copy of the array.  What "const" does is simply tell the compiler that making a copy of the array is not necessary since the array elements are not modified.

IOW, the array is always passed by reference.  You can see that in the generated assembly.
Title: Re: I need clarification with "open Arrays"
Post by: Thaddy on August 18, 2019, 03:18:10 pm
Rewrite InitArray as a function that return an array.
That's very inefficient.  The compiler will allocate stack space to hold the result of the function then copy the result into the receiving variable.



ETA:

@Thaddy

Only in the case of a const open array
No.  Even if the array is passed by value, the compiler will pass the array by reference and the reference is used to make a copy of the array.  What "const" does is simply tell the compiler that making a copy of the array is not necessary since the array elements are not modified.
IOW, the array is always passed by reference.  You can see that in the generated assembly.

That was not the point. The point is that an open array declared as const can not be modified, whereas a typed dynamic array declared as const can be modified!.
That's why I prefer typed dynamic arrays over open arrays declared as const in a parameter list.

I suspect again an intentional misrepresentation of my answer.
Title: Re: I need clarification with "open Arrays"
Post by: 440bx on August 18, 2019, 03:46:42 pm
I suspect again an intentional misrepresentation of my answer.
No.  There was no intentional misrepresentation of your answer.

...whereas a typed dynamic array declared as const can be modified!.
Can you give an example of a "typed dynamic array declared as const" whose elements can be modified ?.  I doubt the compiler is going to allow modifying the elements of an array that has been declared as const parameter to some function/procedure.

Whether the array is dynamic or static, it will treated exactly the same way when it is passed as a parameter that takes an open array.



Title: Re: I need clarification with "open Arrays"
Post by: ASerge on August 18, 2019, 05:43:50 pm
Just for precision's sake.  The array will always be passed by reference.  The presence of "const" tells the compiler that it doesn't need to make a copy of the array (since the values won't/cannot be changed/assigned to.)  If "const" is not specified then a copy of the array is made and all array operations are done on the copy.
Yes. Rather, it is implementation details, and in the documentation of FPC I didn't see it. Although Delphi has Passing Parameters (http://docwiki.embarcadero.com/RADStudio/Rio/en/Program_Control_(Delphi)#Passing_Parameters)
Title: Re: I need clarification with "open Arrays"
Post by: jamie on August 18, 2019, 05:54:20 pm
I did some Assembler views and yes the array is always passed via reference pointer however, the reference points to ether the original table if using "Var" or it points to a table that was copied over into stack land but there is always a pointer reference in the function parameters ether way.
 
 A CONST in this case seems to always point to the original table, basically the same a VAR but does not allow the coder of course to modify it.

 So no matter how you look at it, not using a VAR causes a waste of CPU time and space that can be used for something else.

 I am glad I got this clarified because I've seen lots of usages of Array parameters not using CONST or VAR.

 I did a time loop study and there is definitely a difference, a large one depending on the size of the table.
Title: Re: I need clarification with "open Arrays"
Post by: Thaddy on August 18, 2019, 06:09:23 pm
Can you give an example of a "typed dynamic array declared as const" whose elements can be modified ?.  I doubt the compiler is going to allow modifying the elements of an array that has been declared as const parameter to some function/procedure.
I actually just did! You are really useless today, aren't you? :) O:-) https://forum.lazarus.freepascal.org/index.php/topic,46444.msg330997.html#msg330997

See that I modify a value to 100. It also can be retrieved outside .
See also that you can not do that with the commented code in the first overload.

Here it just means you can not modify the size, but you can modify the individual content of the elements. (Delphi compatible) There are no copy semantics there.
Title: Re: I need clarification with "open Arrays"
Post by: 440bx on August 19, 2019, 01:10:23 am
I actually just did! You are really useless today, aren't you? :) O:-)
That thing you call an "example" doesn't even compile therefore it is like you, eternally useless.

But, knowing you, I should have specified you provide a _working_ example. 
Title: Re: I need clarification with "open Arrays"
Post by: PascalDragon on August 19, 2019, 09:30:08 am
I actually just did! You are really useless today, aren't you? :) O:-)
That thing you call an "example" doesn't even compile therefore it is like you, eternally useless.
It requires FPC 3.2 or newer, because of the initialization of a and b.
Title: Re: I need clarification with "open Arrays"
Post by: 440bx on August 19, 2019, 09:39:47 am
It requires FPC 3.2 or newer, because of the initialization of a and b.
Thank you for that information.

On the subject of FPC 3.2, any idea as to when it might be released ?
Title: Re: I need clarification with "open Arrays"
Post by: PascalDragon on August 19, 2019, 09:46:23 am
Hopefully this year...  :-[