Recent

Author Topic: Is it safe to store the address of a dynamic array element  (Read 1234 times)

Nimral

  • Full Member
  • ***
  • Posts: 176
  • Keep it simple.
This night I was plagued by a nightmare. I have used a structure like this to store a bunch of records:

Code: Pascal  [Select][+][-]
  1. type
  2.   PPayload = ^TPayload;
  3.  
  4.   TPayload = record
  5.     ID: integer;
  6.     Name: string[50];
  7.   end;
  8.   TArrPayload = array of TPayload;
  9.  
  10. var
  11.   ArrPayload: TArrPayload;
  12.  

Later on I store the address of each element of the array as a pointer in a Virtual Treeview, and finally the tree captions are read from the array by the VirtualStringTree code.

Code: Pascal  [Select][+][-]
  1.   for i := low(ArrPayload) to high(ArrPayload) do
  2.   begin
  3.     VirtualStringTree1.AddChild(nil, @ArrPayload[i]);
  4.   end;
  5.  
  6. ---
  7.  
  8.   assert(assigned(Node), 'VirtualStringTree1GetText: Node not assigned');
  9.   // retrieve the address of the record
  10.   p := PPayload(Sender.GetNodeData(Node)^);
  11.   CellText := p^.Name;
  12.  
  13.  

Tonight it came to me that I dimly remember having read that dynamic arrays are stored in a consecutive memory block, so when I add elements it might happen that the whole array needs to be relocated to a different memory block large enough to hold the whole array.

Is this correct? Do I have to expect that the VST breaks as I add array elements, because at some point the array gets moved to a larger memory block, invalidating all the stored pointers?

Thnx, Armin.
« Last Edit: May 07, 2021, 09:23:01 am by Nimral »
Lazarus 2.0.12 & 2.3.0  (trunk) on Windows 10, Raspberry Pi OS "Buster", macOS Catalina, macOS BigSur, VMWare Workstation 15

wp

  • Hero Member
  • *****
  • Posts: 8579
Re: Is it safe to store the address of a dynamic array element
« Reply #1 on: May 07, 2021, 10:26:04 am »
Do I have to expect that the VST breaks as I add array elements, because at some point the array gets moved to a larger memory block, invalidating all the stored pointers?
I did not test it, but I think so. I'd use a TList or similar rather than an array to store the items - see attachment.
Mainly Lazarus trunk / fpc 3.2.0 / all 32-bit on Win-10, but many more...

MarkMLl

  • Hero Member
  • *****
  • Posts: 2875
Re: Is it safe to store the address of a dynamic array element
« Reply #2 on: May 07, 2021, 12:21:59 pm »
In the general case, i.e. considering multiple languages and architectures, no.

It's safe to store a reference that you've been given explicitly and is documented as safe, like the reference that transparently refers to an object or an entire string or dynamic array, or to storage space that you've requested on the heap (which won't be moved or resized unless you request that).

It's not safe to store an address that you've finagled out of the runtimes, such as the address of one specific dynamic array element or one specific character in a string, because changes in the size of a dynamic array etc. might demand that its elements are moved.

If changes to the size of a dynamic array or string do not (ever) result in its elements being moved, then its storage must be fragmented... hence you can't rely on pointer arithmetic.

Assume that somebody designing a collection object (list, dictionary etc.) is aware of the pitfalls, and stores reference[index] of something he's looking at rather than address.

MarkMLl
Turbo Pascal v1 on CCP/M-86, multitasking with LAN and graphics in 128Kb.
Pet hate: people who boast about the size and sophistication of their computer.
GitHub repositories: https://github.com/MarkMLl?tab=repositories

Martin_fr

  • Administrator
  • Hero Member
  • *
  • Posts: 7311
  • Debugger - SynEdit - and more
    • wiki
Re: Is it safe to store the address of a dynamic array element
« Reply #3 on: May 07, 2021, 12:57:08 pm »
Do I have to expect that the VST breaks as I add array elements, because at some point the array gets moved to a larger memory block, invalidating all the stored pointers?
I did not test it, but I think so. I'd use a TList or similar rather than an array to store the items - see attachment.

TList also re-allocates it's memory. Not on every add, but eventually.
If you know the upper max bound, you can set the capacity first. But then you can pre-alloc your dyn array too.

If you need to keep track of your items, allocate them one by one with GetMem or New, and either:
- add their address to a list/array
- create a linked list

egsuh

  • Hero Member
  • *****
  • Posts: 785
Re: Is it safe to store the address of a dynamic array element
« Reply #4 on: May 07, 2021, 11:47:21 pm »
Quote
Do I have to expect that the VST breaks as I add array elements, because at some point the array gets moved to a larger memory block, invalidating all the stored pointers?

I guess this may happen.  I do not know how VST operates, but

          VST.AddChild(parentnode, pointer); 

seems to allocate memory block (using GetMem (resultnode, recordsize))  if pointer is not defined, but if pointer is defined then it just stores the address internally.

In this kind of structure, the default is that you do not keep the record data yourself (ArrPayLoad in your case). You find a node in someway, and just retrieve the data in it. But sometimes it is necessary to maintain the data outside VST.

So I recommend you to keep array of pointers to record, not record data themselves, as I have shown in another thread. In this way, even though the physical address of ArrPPayLoad (which is array of PPayLoad) is changed, the address of records stored in the ArrPPayLoad and VST themselves are not changed.

Hope my comment is not offensive to you... I'm answering based on assumptions and more than you asked.

 

TinyPortal © 2005-2018