Recent

Author Topic: Dynamic arrays are automatically deallocated?  (Read 12212 times)

sam707

  • Guest
Re: Dynamic arrays are automatically deallocated?
« Reply #15 on: September 24, 2017, 06:06:40 pm »
STRINGS ARE NOT DYNAMIC ARRAYS

strings are containers TO special dynamic arrays, references counted. many strings can point to ONE dynamic array of chars as long as you did not use the UniqueString procedure

dynamic arrays can contain much more than bytes or chars => complex type like records or objects for example

think that "string" is a high level Envelop around a special kind of RESTRICTIVE dynarray (chars only with only one dimension)
« Last Edit: September 24, 2017, 06:13:12 pm by sam707 »

howardpc

  • Hero Member
  • *****
  • Posts: 4144
Re: Dynamic arrays are automatically deallocated?
« Reply #16 on: September 24, 2017, 08:42:16 pm »
Why the need to SHOUT?

Strings (except shortstrings of course) are certainly not static arrays, and are managed types analagous to other types the compiler reference counts and manages for you behind the scenes.

Eugene Loza

  • Hero Member
  • *****
  • Posts: 673
    • My games in Pascal
Re: Dynamic arrays are automatically deallocated?
« Reply #17 on: September 24, 2017, 09:15:47 pm »
Is the memory still reserved ?
A bit offtopic:
I recall my first experience in programming. Back in early 90s, on my Poisk-1 computer I've started with GWBasic and then switched to QBasic, before coming to TurboPascal. There was a very nice "cheat" back then. If I didn't save my QBasic project for a long time and the PC would hang up, I could restore some of the code - create a new project and exit QBasic - it'd ask whether to save the unsaved project, and if "YES" it'd recover a part of previous code. Yes, some part of memory content survived MSDOS reboot.
My FOSS games in FreePascal&CastleGameEngine: https://decoherence.itch.io/ (Sources: https://gitlab.com/EugeneLoza)

Tommi

  • Full Member
  • ***
  • Posts: 213
Re: Dynamic arrays are automatically deallocated?
« Reply #18 on: September 24, 2017, 09:49:31 pm »
This policy leads to a lot of possible bugs because your code hasn't the same behavior in any occasion on any PC. Some time your code may work good and you have the false certainty that is all ok.

It would be very useful a directive that says to the compiler to clean to zero all the allocated memory. For example strings should be empty, integer should be 0, dynamic arrays set to zero length etc.
Something like:
{$mode autoZero}

May be that it has the collateral effect of a little waste of time at beginning of the function but it would prevent a lot of bugs.

Another way would be to create a reserved word to use instead "var".

For example:
Code: [Select]
function myFunction(input:integer):string;
autozeroVar
  bb:array of integer;
  cc:array [1..100] of integer;
  a:integer;
begin
  a:=a+cc[1]+2;  //Here I am sure that a is equal to 2. This code every time has the same behavior.
 ..my code..
end;

istead:

Code: [Select]
function myFunction(input:integer):string;
Var
  bb:array of integer;
  cc:array [1..100] of integer;
  a:integer;
begin
  a:=a+cc[1]+2;  //Here I DON'T know the value of a. This code may work or mya not.
 ..my code..
end;

I know that about variables you can do:
Code: [Select]
var
a:integer=0;

But is not the same.
« Last Edit: September 24, 2017, 09:51:48 pm by Tommi »

howardpc

  • Hero Member
  • *****
  • Posts: 4144
Re: Dynamic arrays are automatically deallocated?
« Reply #19 on: September 24, 2017, 10:09:47 pm »
It would be very useful a directive that says to the compiler to clean to zero all the allocated memory. For example strings should be empty, integer should be 0, dynamic arrays set to zero length etc.

The recently introduced Default() intrinsic does what you are asking.
Additionally, all global variables and all fields in instances of classes are initialised to zero by compiler-generated code.
The occasions where you need to do a manual initialisation with Default() or FillChar() or FillWord() are not that many.
Non-shortstring strings are empty, BTW, after declaration.
I don't see a problem with a Setlength(..., 0) when you first declare a dynamic array (although in most cases even that is not needed).
And the compiler warns in almost every case about uninitialised variables.

taazz

  • Hero Member
  • *****
  • Posts: 5368
Re: Dynamic arrays are automatically deallocated?
« Reply #20 on: September 24, 2017, 10:11:25 pm »
This policy leads to a lot of possible bugs because your code hasn't the same behavior in any occasion on any PC. Some time your code may work good and you have the false certainty that is all ok.

It would be very useful a directive that says to the compiler to clean to zero all the allocated memory. For example strings should be empty, integer should be 0, dynamic arrays set to zero length etc.
Something like:
{$mode autoZero}

May be that it has the collateral effect of a little waste of time at beginning of the function but it would prevent a lot of bugs.

Another way would be to create a reserved word to use instead "var".
That is a half measure. What happens when the compiler decides to change the default initialization values to something more mainstream. There is no substitution of proper variable initialization, it is the first lesson one learns in programming, granted usually in passing, but it becomes fast a hard learned lesson. I have spend around 2 man months chasing a bug of, what looked at the time, properly written code only to have it disappear on me on the next compilation. Once I initialized my vars never seen it again.


Do not assume that the compiler will hold your hand make sure your variables have the appropriate starting values.
Good judgement is the result of experience … Experience is the result of bad judgement.

OS : Windows 7 64 bit
Laz: Lazarus 1.4.4 FPC 2.6.4 i386-win32-win32/win64

guest58172

  • Guest
Re: Dynamic arrays are automatically deallocated?
« Reply #21 on: September 24, 2017, 10:13:11 pm »
in D, default init also works on local variables (at least Obj Pascal does it on aggregate members). This means that unless you init a local yourself you're 100% sure of the value it has when you start using it.

Mr.Madguy

  • Hero Member
  • *****
  • Posts: 844
Re: Dynamic arrays are automatically deallocated?
« Reply #22 on: September 25, 2017, 08:14:37 am »
Always check generated code and sources, if you are confused about something. Strings - are basically 1D dynamic arrays with some additional info, attached to them - code page and element size. Everything else is the same - reference count and length. Hmmm. Both should be unique, i.e. if array length is changed and there are more, than 1 reference to this array - data is copied to new array. It's right behavior for strings, but I'm not sure, if this is right for dynamic arrays. I guess, it's done for compatibility with Delphi, right?

See? Dynamic array variable itself is initialized by compiler (zero length just means, that it's = Nil - same, as for strings) and FillChars are everywhere in SetLength. Therefore everything should be initialized. If it doesn't happen - then something is wrong with your code. Some sort of heap corruption may be.

Dynamic array:
Code: Pascal  [Select][+][-]
  1. procedure TForm1.FormCreate(Sender: TObject);
  2.   var A:array of Integer;
  3.   I:Integer;
  4. begin
  5.   SetLength(A, 10);
  6.   for I := Low(A) to High(A) do begin
  7.     A[I] := I;
  8.   end;
  9. end;
  10.  
Code: [Select]
push   ebx
mov    DWORD PTR [ebp-0x8],eax
mov    DWORD PTR [ebp-0x4],edx
mov    DWORD PTR [ebp-0xc],0x0 //Our dynamic array
mov    eax,0x1
lea    edx,[ebp-0x34]
lea    ecx,[ebp-0x1c]
call   0x40d5d0 <fpc_pushexceptaddr>
call   0x40f1c0 <fpc_setjmp>
push   eax
test   eax,eax
jne    0x424579 <TFORM1__FORMCREATE+121>
mov    DWORD PTR [ebp-0x3c],0xa //Length
lea    eax,[ebp-0x3c]
push   eax //Array of dimensions is passed via stack
mov    edx,0x54c4b8 //Type info
lea    eax,[ebp-0xc] //Our dynamic array
mov    ecx,0x1 //Number of dimensions
call   0x40c5e0 <fpc_dynarray_setlength> //SetLength is here
mov    eax,DWORD PTR [ebp-0xc]
call   0x40c4d0 <fpc_dynarray_high>
mov    DWORD PTR [ebp-0x10],0x0
cmp    eax,DWORD PTR [ebp-0x10]
jl     0x424579 <TFORM1__FORMCREATE+121>
sub    DWORD PTR [ebp-0x10],0x1
add    DWORD PTR [ebp-0x10],0x1
mov    edx,DWORD PTR [ebp-0xc]
mov    ecx,DWORD PTR [ebp-0x10]
mov    ebx,DWORD PTR [ebp-0x10]
mov    DWORD PTR [edx+ecx*4],ebx
cmp    eax,DWORD PTR [ebp-0x10]
jg     0x424564 <TFORM1__FORMCREATE+100>
call   0x40d850 <fpc_popaddrstack>
mov    edx,0x54c4b8 //Type info
lea    eax,[ebp-0xc] //Our dynamic array
call   0x40e330 <fpc_finalize> //Deallocation is here
pop    eax
test   eax,eax
je     0x424595 <TFORM1__FORMCREATE+149>
call   0x40d980 <fpc_reraise>
pop    ebx
leave 
ret   

Code: Pascal  [Select][+][-]
  1. procedure fpc_dynarray_setlength(var p : pointer;pti : pointer;
  2.   dimcount : sizeint;dims : pdynarrayindex);[Public,Alias:'FPC_DYNARR_SETLENGTH']; compilerproc;
  3.  
  4.   var
  5.      i : tdynarrayindex;
  6.      movelen,
  7.      size : sizeint;
  8.      { contains the "fixed" pointers where the refcount }
  9.      { and high are at positive offsets                 }
  10.      realp,newp : pdynarray;
  11.      ti : pointer;
  12.      updatep: boolean;
  13.      elesize : sizeint;
  14.      eletype : pointer;
  15.  
  16.   begin
  17.      { negative length is not allowed }
  18.      if dims[0]<0 then
  19.        HandleErrorAddrFrameInd(201,get_pc_addr,get_frame);
  20.  
  21.      { skip kind and name }
  22.      ti:=aligntoptr(Pointer(pti)+2+PByte(pti)[1]);
  23.  
  24.      elesize:=pdynarraytypedata(ti)^.elSize;
  25.      eletype:=pdynarraytypedata(ti)^.elType2;
  26.  
  27.      { determine new memory size }
  28.      size:=elesize*dims[0]+sizeof(tdynarray);
  29.      updatep := false;
  30.  
  31.      { not assigned yet? }
  32.      if not(assigned(p)) then
  33.        begin
  34.           { do we have to allocate memory? }
  35.           if dims[0] = 0 then
  36.             exit;
  37.           getmem(newp,size);
  38.           fillchar(newp^,size,0);
  39.           updatep := true;
  40.        end
  41.      else
  42.        begin
  43.           { if the new dimension is 0, we've to release all data }
  44.           if dims[0]=0 then
  45.             begin
  46.                fpc_dynarray_clear(p,pti);
  47.                exit;
  48.             end;
  49.  
  50.           realp:=pdynarray(p-sizeof(tdynarray));
  51.           newp := realp;
  52.  
  53.           if realp^.refcount<>1 then
  54.             begin
  55.                updatep := true;
  56.                { make an unique copy }
  57.                getmem(newp,size);
  58.                fillchar(newp^,size,0);
  59.                if realp^.high < dims[0] then
  60.                  movelen := realp^.high+1
  61.                else
  62.                  movelen := dims[0];
  63.                move(p^,(pointer(newp)+sizeof(tdynarray))^,elesize*movelen);
  64.  
  65.                { increment ref. count of members }
  66.                for i:= 0 to movelen-1 do
  67.                  int_addref(pointer(newp)+sizeof(tdynarray)+elesize*i,eletype);
  68.  
  69.                { a declock(ref. count) isn't enough here }
  70.                { it could be that the in MT environments  }
  71.                { in the mean time the refcount was       }
  72.                { decremented                             }
  73.  
  74.                { it is, because it doesn't really matter }
  75.                { if the array is now removed             }
  76.                fpc_dynarray_clear(p,pti);
  77.             end
  78.           else if dims[0]<>realp^.high+1 then
  79.             begin
  80.                { range checking is quite difficult ...  }
  81.                { if size overflows then it is less than }
  82.                { the values it was calculated from      }
  83.                if (size<sizeof(tdynarray)) or
  84.                  ((elesize>0) and (size<elesize)) then
  85.                  HandleErrorAddrFrameInd(201,get_pc_addr,get_frame);
  86.  
  87.                { resize? }
  88.                { here, realp^.refcount has to be one, otherwise the previous }
  89.                { if-statement would have been taken. Or is this also for MT  }
  90.                { code? (JM)                                                  }
  91.                if realp^.refcount=1 then
  92.                  begin
  93.                     { shrink the array? }
  94.                     if dims[0]<realp^.high+1 then
  95.                       begin
  96.                           int_finalizearray(pointer(realp)+sizeof(tdynarray)+
  97.                             elesize*dims[0],
  98.                             eletype,realp^.high-dims[0]+1);
  99.                          reallocmem(realp,size);
  100.                       end
  101.                     else if dims[0]>realp^.high+1 then
  102.                       begin
  103.                          reallocmem(realp,size);
  104.                          fillchar((pointer(realp)+sizeof(tdynarray)+elesize*(realp^.high+1))^,
  105.                            (dims[0]-realp^.high-1)*elesize,0);
  106.                       end;
  107.                     newp := realp;
  108.                     updatep := true;
  109.                  end;
  110.             end;
  111.        end;
  112.     { handle nested arrays }
  113.     if dimcount>1 then
  114.       begin
  115.          for i:=0 to dims[0]-1 do
  116.            int_dynarray_setlength(pointer((pointer(newp)+sizeof(tdynarray)+i*elesize)^),
  117.              eletype,dimcount-1,@dims[1]);
  118.       end;
  119.      if updatep then
  120.        begin
  121.          p:=pointer(newp)+sizeof(tdynarray);
  122.          newp^.refcount:=1;
  123.          newp^.high:=dims[0]-1;
  124.        end;
  125.   end;
  126.  
Same for String:
Code: Pascal  [Select][+][-]
  1. procedure TForm1.FormCreate(Sender: TObject);
  2.   var S:String;
  3.   I:Integer;
  4. begin
  5.   SetLength(S, 10);
  6.   for I := Low(S) to High(S) do begin
  7.     S[I] := Chr(I);
  8.   end;
  9. end;
  10.  
Code: [Select]
push   ebx
mov    DWORD PTR [ebp-0x8],eax
mov    DWORD PTR [ebp-0x4],edx
mov    DWORD PTR [ebp-0xc],0x0 //Our string
mov    eax,0x1
lea    edx,[ebp-0x34]
lea    ecx,[ebp-0x1c]
call   0x40d5d0 <fpc_pushexceptaddr>
call   0x40f1c0 <fpc_setjmp>
push   eax
test   eax,eax
jne    0x424577 <TFORM1__FORMCREATE+119>
lea    eax,[ebp-0xc] //Our string
mov    cx,0x0 //Code page (default?)
mov    edx,0xa //Length
call   0x4098b0 <fpc_ansistr_setlength> //SetLength is here
mov    ebx,DWORD PTR [ebp-0xc]
test   ebx,ebx
je     0x42454a <TFORM1__FORMCREATE+74>
mov    ebx,DWORD PTR [ebx-0x4]
mov    DWORD PTR [ebp-0x10],0x1
cmp    ebx,DWORD PTR [ebp-0x10]
jl     0x424577 <TFORM1__FORMCREATE+119>
sub    DWORD PTR [ebp-0x10],0x1
mov    esi,esi
add    DWORD PTR [ebp-0x10],0x1
lea    eax,[ebp-0xc]
call   0x403c10 <fpc_ansistr_unique>
mov    ecx,DWORD PTR [ebp-0x10]
mov    dl,BYTE PTR [ebp-0x10]
mov    BYTE PTR [eax+ecx*1-0x1],dl
cmp    ebx,DWORD PTR [ebp-0x10]
jg     0x42455c <TFORM1__FORMCREATE+92>
call   0x40d850 <fpc_popaddrstack>
lea    eax,[ebp-0xc] //Our string
call   0x403bd0 <fpc_ansistr_decr_ref> //Deallocation is here
pop    eax
test   eax,eax
je     0x42458e <TFORM1__FORMCREATE+142>
call   0x40d980 <fpc_reraise>
pop    ebx
leave 
ret   
Code: Pascal  [Select][+][-]
  1. Procedure fpc_AnsiStr_SetLength (Var S : RawByteString; l : SizeInt{$ifdef FPC_HAS_CPSTRING};cp : TSystemCodePage{$endif FPC_HAS_CPSTRING});[Public,Alias : 'FPC_ANSISTR_SETLENGTH'];  compilerproc;
  2. {
  3.   Sets The length of string S to L.
  4.   Makes sure S is unique, and contains enough room.
  5. }
  6. Var
  7.   Temp : Pointer;
  8.   lens, lena,
  9.   movelen : SizeInt;
  10. begin
  11.   if (l>0) then
  12.     begin
  13.       if Pointer(S)=nil then
  14.         begin
  15.           Pointer(S):=NewAnsiString(L);
  16. {$ifdef FPC_HAS_CPSTRING}
  17.           cp:=TranslatePlaceholderCP(cp);
  18.           PAnsiRec(Pointer(S)-AnsiFirstOff)^.CodePage:=cp;
  19. {$else}
  20.           PAnsiRec(Pointer(S)-AnsiFirstOff)^.CodePage:=DefaultSystemCodePage;
  21. {$endif FPC_HAS_CPSTRING}
  22.         end
  23.       else if PAnsiRec(Pointer(S)-AnsiFirstOff)^.Ref=1 then
  24.         begin
  25.           Temp:=Pointer(s)-AnsiFirstOff;
  26.           lens:=MemSize(Temp);
  27.           lena:=AnsiFirstOff+L+sizeof(AnsiChar);
  28.           { allow shrinking string if that saves at least half of current size }
  29.           if (lena>lens) or ((lens>32) and (lena<=(lens div 2))) then
  30.             begin
  31.               reallocmem(Temp,lena);
  32.               Pointer(S):=Temp+AnsiFirstOff;
  33.             end;
  34.         end
  35.       else
  36.         begin
  37.           { Reallocation is needed... }
  38.           Temp:=NewAnsiString(L);
  39.           PAnsiRec(Pointer(Temp)-AnsiFirstOff)^.CodePage:=PAnsiRec(Pointer(S)-AnsiFirstOff)^.CodePage;
  40.           { also move terminating null }
  41.           lens:=succ(length(s));
  42.           if l<lens then
  43.             movelen:=l
  44.           else
  45.             movelen:=lens;
  46.           Move(Pointer(S)^,Temp^,movelen);
  47.           fpc_ansistr_decr_ref(Pointer(s));
  48.           Pointer(S):=Temp;
  49.         end;
  50.       { Force nil termination in case it gets shorter }
  51.       PByte(Pointer(S)+l)^:=0;
  52.       PAnsiRec(Pointer(S)-AnsiFirstOff)^.Len:=l;
  53.     end
  54.   else  { length=0, deallocate the string }
  55.     fpc_ansistr_decr_ref (Pointer(S));
  56. end;
  57.  
  58. Function NewAnsiString(Len : SizeInt) : Pointer;
  59. {
  60.   Allocate a new AnsiString on the heap.
  61.   initialize it to zero length and reference count 1.
  62. }
  63. Var
  64.   P : Pointer;
  65. begin
  66.   { request a multiple of 16 because the heap manager alloctes anyways chunks of 16 bytes }
  67.   GetMem(P,Len+(AnsiFirstOff+sizeof(char)));
  68.   If P<>Nil then
  69.    begin
  70.      PAnsiRec(P)^.Ref:=1;         { Set reference count }
  71.      PAnsiRec(P)^.Len:=0;         { Initial length }
  72.      PAnsiRec(P)^.CodePage:=DefaultSystemCodePage;
  73.      PAnsiRec(P)^.ElementSize:=SizeOf(AnsiChar);
  74.      inc(p,AnsiFirstOff);         { Points to string now }
  75.      PAnsiChar(P)^:=#0;           { Terminating #0 }
  76.    end;
  77.   NewAnsiString:=P;
  78. end;
  79.  
« Last Edit: September 25, 2017, 10:02:15 am by Mr.Madguy »
Is it healthy for project not to have regular stable releases?
Just for fun: Code::Blocks, GCC 13 and DOS - is it possible?

PascalDragon

  • Hero Member
  • *****
  • Posts: 5469
  • Compiler Developer
Re: Dynamic arrays are automatically deallocated?
« Reply #23 on: September 29, 2017, 10:02:35 pm »
I noted that sometime when I enter in a function where is declared a dynamic array, it remembers the old content.

In cases as below:
Code: Pascal  [Select][+][-]
  1. function myfunction(a:string;b:string):integer;
  2. var
  3.   myArray:array of integer;
  4.   resultingString:string;
  5.   a:integer;
  6. begin
  7.  for a:= 1 to 100 do
  8.  begin
  9.     ... code of the loop...
  10.   setLength(myArray,length(myArray)+1);
  11.   myArray[length(myArray)-1]:=resultingString;
  12.  end;
  13. end;
  14. ... code of the function ...
  15.  

I have to put setLength(myArray,0); at the beginning of the function to have everything working.

Would you please provide a small, self contained and compileable example that shows this problem? Cause the compiler initializes the array to Nil upon entering the function, so I suspect that you're doing something that has a sideeffect on this.
Note: Yes, the compiler warns that myArray is not initialized, but that does not change the fact that it has a well defined value.

Two additional notes:
  • it's generally considered a bad idea to increase array by steps of 1 as that fragments the heap quite a bit; better is to either allocate how many items you'll need in the need or increase the length by 2 and readjust the size at the end (you could also do a larger estimate of the whole array size at the beginning and resize the array at the end)
  • instead of "Length(myArray) - 1" for the index you can also use "High(myArray)"

 

TinyPortal © 2005-2018