* * *

Author Topic: dynamic array bug ?  (Read 1431 times)

Nitorami

  • Sr. Member
  • ****
  • Posts: 344
dynamic array bug ?
« on: June 26, 2018, 04:57:54 pm »
Can someone explain the behaviour of this code ?

Setlength () is expected to set newly allocated array elements to zero. However when applied to "result" from within a function, it initialises result with the contents of the previously processed array !?!?.

Code: Pascal  [Select]
  1. {$mode objfpc}
  2. {$OPTIMIZATION OFF}
  3. {$R+,S+,Q+}
  4.  
  5. type  Vector = array of integer;
  6.  
  7. function NewVector (len: longint): Vector;
  8. begin
  9. //  result := nil; //that solves the issue... but why ?
  10.   SetLength (result,len);
  11. end;
  12.  
  13. procedure Print (const V: Vector);
  14. var i: integer;
  15. begin
  16.   for i := 0 to high(V) do write (V[i]:4);
  17.   writeln;
  18. end;
  19.  
  20. var  A,B: Vector;
  21. begin
  22.   A := NewVector(3);
  23.   A[0] := 1;
  24.   A[1] := 4;
  25.   A[2] := 9;
  26.  
  27.   B := NewVector(3);
  28.  
  29.   writeln ('Address of A is: ',ptrint(A));
  30.   write   ('Content of A is: ');
  31.   Print (A);
  32.  
  33.   writeln ('Address of B is: ',ptrint(B));
  34.   write   ('Content of B is: ');
  35.   Print (B);
  36. end.
  37.  

Output:
Quote
Address of A is: 4897328
Content of A is:    1   4   9
Address of B is: 4897360
Content of B is:    1   4   9

Initialising "result" to nil solves the issue, but I don't know why. Thought that "result" may still contain the address of A, so setlength() would find an existing array at this place and do nothing, i.e. B would be allocated at the same address as A. Although that would be a bug, it would explain that the content of A and B are the same. However the addresses of A and B are different, so B is in fact created as a new instance, however filled with the content of A.

This is very weird. I guess it is a bug, but maybe someone knows better. Windows, FPC 3.0.4.

turrican

  • Full Member
  • ***
  • Posts: 132
  • Pascal is my life.
    • Homepage
Re: dynamic array bug ?
« Reply #1 on: June 26, 2018, 05:05:05 pm »
{$OPTIMIZATION OFF}

Is that preventing to initialize local stack variable maybe?

Nitorami

  • Sr. Member
  • ****
  • Posts: 344
Re: dynamic array bug ?
« Reply #2 on: June 26, 2018, 05:09:11 pm »
It does not matter whether optimisation is ON or OFF, I just chose the most conservative settings to demonstrate it is not related to optimization.

turrican

  • Full Member
  • ***
  • Posts: 132
  • Pascal is my life.
    • Homepage
Re: dynamic array bug ?
« Reply #3 on: June 26, 2018, 05:11:06 pm »
It does not matter whether optimisation is ON or OFF, I just chose the most conservative settings to demonstrate it is not related to optimization.

Sorry, my fault...

howardpc

  • Hero Member
  • *****
  • Posts: 2752
Re: dynamic array bug ?
« Reply #4 on: June 26, 2018, 05:13:19 pm »
Using an uninitialised variable and then finding it contains (or points to) non-random data is not a compiler bug, but a programmer failing. In this simple case your uninitialised Result happens to pick up the 'same' data consistently. But this is an implementation detail, and cannot be relied on. Result is undefined until you assign a value to it.

turrican

  • Full Member
  • ***
  • Posts: 132
  • Pascal is my life.
    • Homepage
Re: dynamic array bug ?
« Reply #5 on: June 26, 2018, 05:13:46 pm »
That can be related to stack registers... but idk why.

turrican

  • Full Member
  • ***
  • Posts: 132
  • Pascal is my life.
    • Homepage
Re: dynamic array bug ?
« Reply #6 on: June 26, 2018, 05:17:16 pm »
Using an uninitialised variable and then finding it contains (or points to) non-random data is not a compiler bug, but a programmer failing. In this simple case your uninitialised Result happens to pick up the 'same' data consistently. But this is an implementation detail, and cannot be relied on. Result is undefined until you assign a value to it.

That's true and is a good practice to initializate variables but in this case (for example) Delphi compiler works fine. I tried it.
« Last Edit: June 26, 2018, 05:42:23 pm by turrican »

Nitorami

  • Sr. Member
  • ****
  • Posts: 344
Re: dynamic array bug ?
« Reply #7 on: June 26, 2018, 05:43:08 pm »
Quote
Using an uninitialised variable and then finding it contains (or points to) non-random data is not a compiler bug, but a programmer failing. In this simple case your uninitialised Result happens to pick up the 'same' data consistently. But this is an implementation detail, and cannot be relied on. Result is undefined until you assign a value to it.

Not agreed in this case. The documentation does not tell me to initialise the argument of Setlength before using it. SetLength itself does the initialisation.


turrican

  • Full Member
  • ***
  • Posts: 132
  • Pascal is my life.
    • Homepage
Re: dynamic array bug ?
« Reply #8 on: June 26, 2018, 05:52:40 pm »
Quote
SetLength itself does the initialisation.

+1

Nitorami

  • Sr. Member
  • ****
  • Posts: 344
Re: dynamic array bug ?
« Reply #9 on: June 26, 2018, 05:57:18 pm »
I agree that the issue may have to do with initialisation, and it is quite possible that "result" within a function body is not automatically initialised for reasons of performance. But if that causes a dynamic array allocation to go wrong, it is a bug. The compiler itself should take care of the initialisation of managed variables. 

Martin_fr

  • Administrator
  • Hero Member
  • *
  • Posts: 4781
    • wiki
Re: dynamic array bug ?
« Reply #10 on: June 26, 2018, 05:57:53 pm »
Not agreed in this case. The documentation does not tell me to initialise the argument of Setlength before using it. SetLength itself does the initialisation.

Well the issue, as far as I understand it, is:

Dynamic arrays and strings are so called managed types. They are always in some kind of initialized state (i.e. either nil, or valid data (old data can be valid)).
Those types are refcounted (they are pointers to data that includes a refcount). If a dynamic array goes out of scope, its refcount is reduced, and it is freed.

so in case of "var a: array of ...", a will be "nil" to start with (in practice, not necessarily documented) . Because there is no old data available, so there is only nil. Then SetLength will initialize the entries, because all entries are new.

Doing
Code: Pascal  [Select]
  1. SetLength(a,2);
  2. a[1]:=99;
  3. SetLength(a,20);
will keep the value of a[1].

In other words:
- SetLenght initializes new entries in the array
- SetLenght does NOT initialize the array itself


Code: Pascal  [Select]
  1. TIntegerArray = array of integer;
  2. Procedure Foo(var a: TIntegerArray)
var param, will get the value passed in. SetLength will keep existing values.

And here is the final bit, that is less known.
For managed types "Result" is internally handled as a var param.

-------------------------
That is the technical bit.

As for what the expected behaviour should be.... I haven't checked all the docs.

But in general, ALL variables should be initialized before use.

I am not aware that managed types are excluded in the doc (in practice they often are, but its about the doc)

As above,  SetLenght initializes the entries, not the array.
So initialization would be
  Result := nil;
  Result := SomeOtherArray;


--- EDIT
https://www.freepascal.org/docs-html/rtl/system/setlength.html
Doesn't say anything about initializing the entries either.
« Last Edit: June 26, 2018, 06:06:18 pm by Martin_fr »

Nitorami

  • Sr. Member
  • ****
  • Posts: 344
Re: dynamic array bug ?
« Reply #11 on: June 26, 2018, 06:07:05 pm »
That sounds reasonable, but it means I would always have to know whether a variable is initialised or not. My understanding so far was that global variables are not automatically initialised, so something like

Code: Pascal  [Select]
  1. var A: TVector;
  2. SetLength (A,3)

would be likely to fail, but in fact it it always worked for me.

EDIT: It may be that the compiler automatically initialises globally defined managed types to nil, but then we have a dangerous inconsistency. It can hardly be expected from the user to know that only in this single case he needs to set the argument of SetLength to nil before using it. Particularlay where the compiler knows quite well that "result" is a dynamic array, and could do the job automatically.


« Last Edit: June 26, 2018, 06:29:42 pm by Nitorami »

engkin

  • Hero Member
  • *****
  • Posts: 2101
Re: dynamic array bug ?
« Reply #12 on: June 26, 2018, 06:41:10 pm »
Might be easier to see the problem with this tiny replacement:
Code: Pascal  [Select]
  1. procedure NewVector_real (var Result: Vector; len: longint);
  2. begin
  3.   SetLength (Result, len);
  4. end;

In the first call Result was nil.
In the second call Result had the address of the previous call. Since it is not nil, its contents are copied.

wk_

  • Newbie
  • Posts: 1
Re: dynamic array bug ?
« Reply #13 on: June 26, 2018, 07:52:25 pm »
Hi,

Looks like a glaring bug to me. BTW: when modelling "result" as an additional parameter, it should be an out parameter, since from a language point of view you cannot pass anything in when calling a function (iow. "result" always is a fresh variable). In addition, when returning from the function, result goes out of scope, hence the value it holds should be invalidated/destroyed (ref count -1). Finally, since in the program the function is once called to assign to variable A and then to variable B, even a procedure with result as var parameter would not lead to the observed result.

Cheers,
 Will

Nitorami

  • Sr. Member
  • ****
  • Posts: 344
Re: dynamic array bug ?
« Reply #14 on: June 26, 2018, 08:43:36 pm »
I think so, too. Filed a bug report id=33912.

 

Recent

Get Lazarus at SourceForge.net. Fast, secure and Free Open Source software downloads Open Hub project report for Lazarus