Recent

Author Topic: I need clarification with "open Arrays"  (Read 708 times)

jamie

  • Hero Member
  • *****
  • Posts: 1893
I need clarification with "open Arrays"
« on: August 18, 2019, 03:07:51 am »
This for example;
Code: Pascal  [Select]
  1. function SearchTable(CodePageArr: array of word; id: cardinal): word;  
  2.  

if the array is very large, is this getting pushed on the stack each time it gets called?

Shouldn't it be a "VAR" reference in there?
 

winni

  • Full Member
  • ***
  • Posts: 170
Re: I need clarification with "open Arrays"
« Reply #1 on: August 18, 2019, 03:18:56 am »
No. Open arrays are only a pointer. The logic with the VAR parameters is broken since Delphi1 - in the pascal logic of N. Wirth.

Winni
« Last Edit: August 18, 2019, 03:20:33 am by winni »

jamie

  • Hero Member
  • *****
  • Posts: 1893
Re: I need clarification with "open Arrays"
« Reply #2 on: August 18, 2019, 03:32:46 am »
That's strange because reading my Delphi  help file, one of my old tools (3) it states that it does follow the VAR. VALUE or CONST and puts this on the stack  I just found a reference on the net for FPC and it also states the same as Delphi.
 
 A VALUE which is what I showed here will make a copy to the stack so that you can make changes to it locally but not effect it globally and so on with the rest.

 Looking at some code in various units that are used in Lazarus and fpc I have been finding questioned code practices..

 Thanks for the heads up, I am going to run a debug session and view some ASSEMBLER code for varication  :)

jamie

  • Hero Member
  • *****
  • Posts: 1893
Re: I need clarification with "open Arrays"
« Reply #3 on: August 18, 2019, 04:15:57 am »
I just ran some test and it does indeed make copies of the array if you don't use the "VAR" pass by Ref.

 So I guess it does follow along.. I am trying to speed up some code and looking around I just happen to see that in a file. Actually I see that used without "Var" in many places.

 I also found out that Fpc 3.0.4 does not support functions that accept Open Arrays to be inlined.
 Not sure if this has been rectified in later versions yet. its not a big deal because it would only help in some cases.

440bx

  • Hero Member
  • *****
  • Posts: 1085
Re: I need clarification with "open Arrays"
« Reply #4 on: August 18, 2019, 04:39:53 am »
I was going to give an example showing that it  does make a copy of the array (unless you specify "var") and in the process, ran into a bug related to the debugger. 

The following small program, if compiled for 64bit and a breakpoint is placed on the program's "begin" (line 25) then an illegal instruction exception occurs.

The problem does not happen in 32bit.

Code: Pascal  [Select]
  1. program _Arrayof;
  2.  
  3. {$LONGSTRINGS ON}
  4.  
  5.  
  6. type
  7.   T500 = array[0..500] of DWORD;
  8.  
  9. var
  10.   A     : T500;
  11.  
  12. procedure InitArray(p : array of DWORD);
  13. var
  14.   i : integer;
  15. begin
  16.   for i := low(p) to high(p) do
  17.   begin
  18.     p[i] := 7;    // any value
  19.   end;
  20. end;
  21.  
  22. var
  23.   i : integer;
  24.  
  25. begin
  26.   InitArray(A);
  27.  
  28.   for i := low(A) to high(A) do
  29.   begin
  30.     writeln('A[', i, '] = ', A[i]);
  31.   end;
  32.  
  33. end.
  34.  
« Last Edit: August 18, 2019, 04:41:38 am by 440bx »
using FPC v3.0.4 and Lazarus 1.8.2 on Windows 7 64bit.

WooBean

  • Jr. Member
  • **
  • Posts: 89
Re: I need clarification with "open Arrays"
« Reply #5 on: August 18, 2019, 08:01:10 am »
Confirmed for FPC 3.04 Lazarus 1.8 (win 32 bit) - code executes, for FPC 3.04 Lazarus 2.04 (win 64 bit) - code fails.
More, if "procedure InitArray(p : array of DWORD);"  is declared as "procedure InitArray(var p : array of DWORD);" it works in both environments.     
My comments is that there are differences at accepting static arrays by FPC 3.0.4 as open array depending on usage of var keyword.

WooBean
Win7/64, Lazarus 1.8 win64-win64, FPC 3.0.4

ASerge

  • Hero Member
  • *****
  • Posts: 1388
Re: I need clarification with "open Arrays"
« Reply #6 on: August 18, 2019, 11:52:52 am »
Code: Pascal  [Select]
  1. function SearchTable(CodePageArr: array of word; id: cardinal): word;  
  2.  
if the array is very large, is this getting pushed on the stack each time it gets called?
In Delphi typically used const, which notifies the compiler that it can optimize the code and pass only the reference, otherwise it pass by value. In FPC to accurately ensure this used constref.
By the way, may be use System.IndexWord?

440bx

  • Hero Member
  • *****
  • Posts: 1085
Re: I need clarification with "open Arrays"
« Reply #7 on: August 18, 2019, 01:00:50 pm »
In Delphi typically used const, which notifies the compiler that it can optimize the code and pass only the reference, otherwise it pass by value.
Just for precision's sake.  The array will always be passed by reference.  The presence of "const" tells the compiler that it doesn't need to make a copy of the array (since the values won't/cannot be changed/assigned to.)  If "const" is not specified then a copy of the array is made and all array operations are done on the copy.




using FPC v3.0.4 and Lazarus 1.8.2 on Windows 7 64bit.

LemonParty

  • New Member
  • *
  • Posts: 28
Re: I need clarification with "open Arrays"
« Reply #8 on: August 18, 2019, 01:36:50 pm »
Quote
Dynamic arrays are reference counted. Calling setLength(myDynamicArrayVariable, 0) virtually does myDynamicArrayVariable := nil and decreases the reference count. Only when the reference count hits zero, the memory block is released.
Rewrite InitArray as a function that return an array.
« Last Edit: August 18, 2019, 01:41:21 pm by LemonParty »

Thaddy

  • Hero Member
  • *****
  • Posts: 8671
Re: I need clarification with "open Arrays"
« Reply #9 on: August 18, 2019, 01:48:03 pm »
In Delphi typically used const, which notifies the compiler that it can optimize the code and pass only the reference, otherwise it pass by value.
Just for precision's sake.  The array will always be passed by reference.  The presence of "const" tells the compiler that it doesn't need to make a copy of the array (since the values won't/cannot be changed/assigned to.)  If "const" is not specified then a copy of the array is made and all array operations are done on the copy.

Only in the case of a const open array e.g:
Code: Pascal  [Select]
  1. {$mode delphi}{$H+}
  2. type
  3.   TWordArray = array of word;
  4.  
  5.   // open array
  6.   function ArrayContains(const arr:array of word;item:word):Boolean;overload;
  7.   var i:integer;
  8.   begin
  9.     Result := false;
  10.     for i :=0 to high(arr) do
  11.       if arr[i] = item then result := true;;
  12. //      if arr[i] = item then begin result := true;arr[i]:=100;exit end; // this won't work
  13.   end;
  14.  
  15.   // typed array
  16.   function ArrayContains(const arr:TWordArray;item:word):Boolean;overload;
  17.   var i:integer;
  18.   begin
  19.      Result := false;
  20.     for i :=0 to high(arr) do
  21.       if arr[i] = item then begin result := true;arr[i]:=100;exit end; // but this will allow the modification
  22.   end;
  23.  
  24. var
  25.   a:TWordArray = [1,2,3,4,5,6,7,8,9];  
  26.   b:array of word = [1,2,3,4,5,6,7,8,9];
  27. begin
  28.   writeln(arrayContains(a,3));
  29.   writeln(arrayContains(b,3));
  30. end.
So if you need to modify declare an array type.
Most people that want to use threading should learn to patch their jeans first: use a needle.

440bx

  • Hero Member
  • *****
  • Posts: 1085
Re: I need clarification with "open Arrays"
« Reply #10 on: August 18, 2019, 02:07:31 pm »
Rewrite InitArray as a function that return an array.
That's very inefficient.  The compiler will allocate stack space to hold the result of the function then copy the result into the receiving variable.



ETA:

@Thaddy

Only in the case of a const open array
No.  Even if the array is passed by value, the compiler will pass the array by reference and the reference is used to make a copy of the array.  What "const" does is simply tell the compiler that making a copy of the array is not necessary since the array elements are not modified.

IOW, the array is always passed by reference.  You can see that in the generated assembly.
« Last Edit: August 18, 2019, 02:15:22 pm by 440bx »
using FPC v3.0.4 and Lazarus 1.8.2 on Windows 7 64bit.

Thaddy

  • Hero Member
  • *****
  • Posts: 8671
Re: I need clarification with "open Arrays"
« Reply #11 on: August 18, 2019, 03:18:10 pm »
Rewrite InitArray as a function that return an array.
That's very inefficient.  The compiler will allocate stack space to hold the result of the function then copy the result into the receiving variable.



ETA:

@Thaddy

Only in the case of a const open array
No.  Even if the array is passed by value, the compiler will pass the array by reference and the reference is used to make a copy of the array.  What "const" does is simply tell the compiler that making a copy of the array is not necessary since the array elements are not modified.
IOW, the array is always passed by reference.  You can see that in the generated assembly.

That was not the point. The point is that an open array declared as const can not be modified, whereas a typed dynamic array declared as const can be modified!.
That's why I prefer typed dynamic arrays over open arrays declared as const in a parameter list.

I suspect again an intentional misrepresentation of my answer.
Most people that want to use threading should learn to patch their jeans first: use a needle.

440bx

  • Hero Member
  • *****
  • Posts: 1085
Re: I need clarification with "open Arrays"
« Reply #12 on: August 18, 2019, 03:46:42 pm »
I suspect again an intentional misrepresentation of my answer.
No.  There was no intentional misrepresentation of your answer.

...whereas a typed dynamic array declared as const can be modified!.
Can you give an example of a "typed dynamic array declared as const" whose elements can be modified ?.  I doubt the compiler is going to allow modifying the elements of an array that has been declared as const parameter to some function/procedure.

Whether the array is dynamic or static, it will treated exactly the same way when it is passed as a parameter that takes an open array.



using FPC v3.0.4 and Lazarus 1.8.2 on Windows 7 64bit.

ASerge

  • Hero Member
  • *****
  • Posts: 1388
Re: I need clarification with "open Arrays"
« Reply #13 on: August 18, 2019, 05:43:50 pm »
Just for precision's sake.  The array will always be passed by reference.  The presence of "const" tells the compiler that it doesn't need to make a copy of the array (since the values won't/cannot be changed/assigned to.)  If "const" is not specified then a copy of the array is made and all array operations are done on the copy.
Yes. Rather, it is implementation details, and in the documentation of FPC I didn't see it. Although Delphi has Passing Parameters

jamie

  • Hero Member
  • *****
  • Posts: 1893
Re: I need clarification with "open Arrays"
« Reply #14 on: August 18, 2019, 05:54:20 pm »
I did some Assembler views and yes the array is always passed via reference pointer however, the reference points to ether the original table if using "Var" or it points to a table that was copied over into stack land but there is always a pointer reference in the function parameters ether way.
 
 A CONST in this case seems to always point to the original table, basically the same a VAR but does not allow the coder of course to modify it.

 So no matter how you look at it, not using a VAR causes a waste of CPU time and space that can be used for something else.

 I am glad I got this clarified because I've seen lots of usages of Array parameters not using CONST or VAR.

 I did a time loop study and there is definitely a difference, a large one depending on the size of the table.