* * *

Author Topic: How to declare a variable absolute with respect to a dynamic array?  (Read 2331 times)

Lupp

  • New member
  • *
  • Posts: 30
What I actually get passed to a procedure for a var parameter of a dynamic array type when replacing it with an actual dynamic array is obviously a pointer into the heap.

To do some work efficiently on a level of byte manipulation, I would like to declare a differently typed variable absoulte with respect to the data space of the (actual) parameter.
What exactly is kept in RAM for a dynamic array and in what way is this organized?

Is it possible to declare a pocedure in a way that it will accept actual dynamic arrays of different element types without writing tons of declarations for overloaded procedures with a common name?

You see, I like strict typing, but the overload concept allows anyway to weaken it.  In a few well considered cases I would like to do so without lots of formalities.

ASerge

  • Sr. Member
  • ****
  • Posts: 313
Re: How to declare a variable absolute with respect to a dynamic array?
« Reply #1 on: February 17, 2017, 08:11:43 pm »
Is it possible to declare a procedure in a way that it will accept actual dynamic arrays of different element types without writing tons of declarations for overloaded procedures with a common name?
You can use generics, like this:
Code: Pascal  [Select]
  1. program Project1;
  2.  
  3. {$APPTYPE CONSOLE}
  4.  
  5. type
  6.   generic TGArray<_T> = class(TObject)
  7.   public
  8.     class procedure SumToFirst(var A: array of _T); static;
  9.   end;
  10.  
  11. class procedure TGArray.SumToFirst(var A: array of _T);
  12. var
  13.   X: _T;
  14.   Sum: _T;
  15. begin
  16.   Sum := Default(_T);
  17.   for X in A do
  18.     Sum := Sum + X;
  19.   A[Low(A)] := Sum;
  20. end;
  21.  
  22. procedure Test;
  23. type
  24.   TIntArray = specialize TGArray<Integer>;
  25.   TStrArray = specialize TGArray<string>;
  26. var
  27.   IA: array of Integer;
  28.   SA: array of string;
  29. begin
  30.   SetLength(IA, 2);
  31.   IA[0] := 1;
  32.   IA[1] := 2;
  33.   SetLength(SA, 2);
  34.   SA[0] := 'one';
  35.   SA[1] := 'two';
  36.   TIntArray.SumToFirst(IA);
  37.   TStrArray.SumToFirst(SA);
  38.   WriteLn(IA[0], ' ', SA[0]);
  39. end;
  40.  
  41. begin
  42.   Test;
  43.   Readln;
  44. end.

molly

  • Hero Member
  • *****
  • Posts: 1619
Re: How to declare a variable absolute with respect to a dynamic array?
« Reply #2 on: February 17, 2017, 08:18:59 pm »
Is it possible to declare a pocedure in a way that it will accept actual dynamic arrays of different element types without writing tons of declarations for overloaded procedures with a common name?
something like array of const you meant ? or perhaps even variants ?
« Last Edit: February 17, 2017, 08:31:22 pm by molly »

howardpc

  • Hero Member
  • *****
  • Posts: 2160
Re: How to declare a variable absolute with respect to a dynamic array?
« Reply #3 on: February 17, 2017, 09:22:13 pm »
You can use an untyped var parameter to pass 'generic' dynamic arrays to a procedure, as in the following example:

Code: Pascal  [Select]
  1. program project1;
  2.  
  3. {$Mode objfpc}{$H+}
  4.  
  5. type
  6.   TStringArray = array of string;
  7.   TIntegerArray = array of integer;
  8.   TDoubleArray = array of double;
  9.  
  10. procedure ShowArray(var anArray; aType: TVarRec);
  11. var
  12.   sa: TStringArray;
  13.   ia: TIntegerArray;
  14.   da: TDoubleArray;
  15.   i, maxi: integer;
  16. begin
  17.   case aType.VType of
  18.     vtAnsiString: begin
  19.       sa:=TStringArray(anArray);
  20.       maxi:=High(sa);
  21.       for i:=0 to maxi do
  22.         if (i <> maxi) then
  23.           Write(sa[i],', ')
  24.         else WriteLn(sa[i]);
  25.     end;
  26.     vtInteger: begin
  27.       ia:=TIntegerArray(anArray);
  28.       maxi:=High(ia);
  29.       for i:=0 to maxi do
  30.         if (i <> maxi) then
  31.           Write(ia[i],', ')
  32.         else WriteLn(ia[i]);
  33.     end;
  34.     vtExtended: begin
  35.       da:=TDoubleArray(anArray);
  36.       maxi:=High(da);
  37.       for i:=0 to maxi do
  38.         if (i <> maxi) then
  39.           Write(da[i]:4:2,', ')
  40.         else WriteLn(da[i]:4:2);
  41.     end;
  42.   end;
  43. end;
  44.  
  45. var
  46.   sa: TStringArray;
  47.   ia: TIntegerArray;
  48.   da: TDoubleArray;
  49.   vr: TVarRec;
  50.  
  51. begin
  52.   SetLength(sa, 4);
  53.   sa:=TStringArray.Create('zero','one','two','three');
  54.   vr.VType:=vtAnsiString;
  55.   ShowArray(sa, vr);
  56.  
  57.   SetLength(ia, 4);
  58.   ia:=TIntegerArray.Create(0, 1, 2, 3);
  59.   vr.VType:=vtInteger;
  60.   ShowArray(ia, vr);
  61.  
  62.   SetLength(da, 4);
  63.   da:=TDoubleArray.Create(0.5, 1.5, 2.5, 3.5);
  64.   vr.VType:=vtExtended;
  65.   ShowArray(da, vr);
  66.  
  67.   WriteLn;
  68.   WriteLn('Press Enter to exit');
  69.   ReadLn;
  70. end.

Lupp

  • New member
  • *
  • Posts: 30
Re: How to declare a variable absolute with respect to a dynamic array?
« Reply #4 on: February 18, 2017, 01:29:25 am »
(My experiences with Pascal were based on Turbo Pascal long ago. I did a bit of programming with Delph 1/2 and very few with 7, subsequently I had a look now and then into very old versions of FreePascal and Lazarus.)

Thanks to all of you. As I already stated, I lack experience with some of the modern concepts. I will have to study your suggestions. 
(I would surely like something like "types as values of variables" if available handily.)

What I wanted to achieve here primarily was rather simple, however:

The var parameter of a procedure, say v, it may be typed TaSmallint (type TaSmallint = array of Smallint) e.g, gives access to the data. It also comes with information about the length of the array set by SetLength.
Suppose I now want to analyze/manipulate the elements bytewise. I declare
Code: Pascal  [Select]
  1. b: array of Byte absolute v;
But now the High(v) value is also applied to b. If I use {$R+} access to the single bytes of the upper half of my Smallints will fail due to violation of the index range. An attempt to resize b would probably move the b-array to elewhere on the heap.

Is there a simple way  to get the indices for b rangechecked as if High(b)=High(v)*2+1 ? The compiler an my program as well have the information needed to calculate the maximum length of b usable without violating the range v brought in.

The behaviour I described does not cange if v is untyped. The crude example
Code: Pascal  [Select]
  1. procedure Test(var v);
  2. {$R-}
  3. var
  4.   a: TaSmall absolute v;
  5.   b: TaByte  absolute v;
  6.   j: Longint;
  7. begin
  8. WriteLn(High(a), High(b):5);
  9. for j:=0 to 9 do
  10.   WriteLn(a[j], b[j*2]:6, b[j*2+1]:6, b[j*2]+b[j*2+1]*256:8);
  11. end;
  12.  
again assumes the same High() value for a and b.

Thaddy

  • Hero Member
  • *****
  • Posts: 3697
Re: How to declare a variable absolute with respect to a dynamic array?
« Reply #5 on: February 18, 2017, 08:58:11 am »
In the case of dynamic arrays there is no real simple way like for static arrays AND maintain range checks..
Due to the nature of dynamic arrays, even if b is declared absolute to a, on setlength() they become two distinct arrays. (see docs)
There are many options, though:
1.- My favorite: write a type helper for one of the array types that has methods to represent it as another type. (AsByte,AsWord,etc)
2.- What I used to do and now strongly discourage: manipulate the array descriptor directly. (Just because it can be done does not mean it should be done)
3.- Use a pointer type to array and calculate Low/High yourself from the first array low/high. What is also what is in 1..
« Last Edit: February 18, 2017, 09:09:00 am by Thaddy »
Why do the Danish always try to fuck up any programming language?

Lupp

  • New member
  • *
  • Posts: 30
Re: How to declare a variable absolute with respect to a dynamic array?
« Reply #6 on: February 18, 2017, 11:46:19 am »
Thanks again.
@Thaddy: You surely know the technical details of the implementation of dynamic arrays with FP and of interoperability issues concerning data exchange in a world also using different programming tools to some degree.

Where and in what way is the High information stored?
Does the "structure" of dynamic arrays also care about the element size? If so, in what way? If not, why?
Does the "structure" of dynamic arrays also provide means to store/make accessible additional metadata?
Is there a text describing this and related topics in a way an amateur like me might understand without intensely studying terminology first?

(Due to hubris I dared to assume the High information for a dynamic array a should be stored some few bytes below the dataspace starting with a[0] then. If so, is there a way to extend the range for metadata downwards?)

Regards and thanks in advance.

(Editing.)
Testing with
Code: Pascal  [Select]
  1. procedure Test(var v);
  2. var
  3.   p1: Pointer absolute v;
  4.   p2: ^Longint;
  5.   a: TaSmall absolute v;
  6.   b: TaByte  absolute v;
  7.   j: Longint;
  8. begin
  9. p2:=p1-8;
  10. WriteLn(p2^);
  11. for j:=0 to High(v) do
  12.   {Something};
  13. end;  

resulted in favor of my assumption with 8 Byte being the size of High information. (Everybody else surely knew that in advance.)
« Last Edit: February 18, 2017, 12:11:33 pm by Lupp »

Thaddy

  • Hero Member
  • *****
  • Posts: 3697
Re: How to declare a variable absolute with respect to a dynamic array?
« Reply #7 on: February 18, 2017, 01:51:52 pm »
Oh, well. I said I discourage that! >:( But you are on the right track.
Some more information:
- You need to keep in mind the native pointer size, so it's not always 8.... (64 pit systems have 64 bit pointers) of valid information
- You need to test for the alignment used, so it can be a lot more than 8...and can be more than 8 in total information .
- Dynamic arrays also have a refcount in the descriptor
But indeed the array descriptor is at a negative offset - and not portable as you may have guessed from the above .... O:-)

This is implementation detail and can change between ANY point release of FPC. It isn't even the same as in Delphi, which also differs between versions.
Stricktly for hacking purposes and not for production code.
« Last Edit: February 18, 2017, 02:02:43 pm by Thaddy »
Why do the Danish always try to fuck up any programming language?

howardpc

  • Hero Member
  • *****
  • Posts: 2160
Re: How to declare a variable absolute with respect to a dynamic array?
« Reply #8 on: February 18, 2017, 01:54:48 pm »
The crude example
Code: Pascal  [Select]
  1. procedure Test(var v);
  2. {$R-}
  3. var
  4.   a: TaSmall absolute v;
  5.   b: TaByte  absolute v;
  6.   j: Longint;
  7. begin
  8. WriteLn(High(a), High(b):5);
  9. for j:=0 to 9 do
  10.   WriteLn(a[j], b[j*2]:6, b[j*2+1]:6, b[j*2]+b[j*2+1]*256:;
  11. end;
  12.  

I'm guessing, but are you after the following?
Code: Pascal  [Select]
  1. program project1;
  2.  
  3. {$Mode objfpc}{$H+}
  4.  
  5. type
  6.   TSmallIntArray = array of SmallInt;
  7.  
  8.   procedure Test(anArray: TSmallIntArray);
  9.   var
  10.     s: SmallInt;
  11.   begin
  12.     for s in anArray do
  13.       WriteLn('s:',s:2,' hi(s):',hi(s):3,' lo(s):',lo(s));
  14.   end;
  15.  
  16. var
  17.   sma: TSmallIntArray;
  18.  
  19. begin
  20.   SetLength(sma, 7);
  21.   sma:=TSmallIntArray.Create(-3,-2,-1,0,1,2,3);
  22.   Test(sma);
  23.  
  24.   WriteLn;
  25.   WriteLn('Press Enter to exit');
  26.   ReadLn;
  27. end.

Thaddy

  • Hero Member
  • *****
  • Posts: 3697
Re: How to declare a variable absolute with respect to a dynamic array?
« Reply #9 on: February 18, 2017, 02:01:35 pm »
Actually what he wants is two types of dynamic array using the same memory space, just like  two static arrays, one of which is declared absolute to the other's address.
That can only somewhat be achieved by any of the methods I provided, since after a setlength the arrays become distinctive.
But you can read (Low() (That is actually always zero for dynamic arrays anyway) and ) High() from the first array, use a pointer, and then use the memory space as if it was an array of a different type. Get it? Sort of absolute for dynamic arrays. (Yes you can declare them absolute, but that won't work after a setlength()......)
Do not use it... can be done... I discourage it... Use at your own risk.

The best solution for his problem is to write an array helper. That can be type safe and memory safe.
« Last Edit: February 18, 2017, 02:06:54 pm by Thaddy »
Why do the Danish always try to fuck up any programming language?

Lupp

  • New member
  • *
  • Posts: 30
Re: How to declare a variable absolute with respect to a dynamic array?
« Reply #10 on: February 18, 2017, 02:36:48 pm »
He (that's me) is aiming
1) at a better understanding.
2) at a few things I consider to write libraries for (to compile to dll for Win, e.g.).
3) at doing so minimizing overhead.
4) at getting high efficiency.
5) not at resizable arrays. This is about dynamic arrays created once in another place and passed to procedures as transients to work on them.

I am aware (hope so) of the issues concerning dependencies on FP version and OS. (In addition calling conventions.) As there obviously not is access to an implementation independent view of array descriptors, not even concerning ElementCount / High and ElementSize, I might either design a class providing this in it's own way, or write procedures directly getting passed the needed information via dedicated parameters. I now will try to do it the second way concerning a first example.

Thaddy

  • Hero Member
  • *****
  • Posts: 3697
Re: How to declare a variable absolute with respect to a dynamic array?
« Reply #11 on: February 18, 2017, 02:55:06 pm »
5) is a dynamic array and therefor by definition a resizable array in pascal.
Alas we have no syntax for it like in C, but we have alternative methods.
Note this is also not in Kernigan and Richie C, the mother of all C(++)..

I think I understand your issue, but this is as it stands.
Just like in C we can manipulate a memory area in any way we feel, but not with a really simple construct.
And C doesn't have High() anyway...
C == un-C-afe by default. If you make many things look easy, you can expect many accidents.
A type helper is just about the best I can think of and that could be made a part of the RTL if sufficiently mature and still respects Pascal efforts to make things safe.
« Last Edit: February 18, 2017, 02:57:44 pm by Thaddy »
Why do the Danish always try to fuck up any programming language?

Lupp

  • New member
  • *
  • Posts: 30
Re: How to declare a variable absolute with respect to a dynamic array?
« Reply #12 on: February 18, 2017, 06:16:32 pm »
Quote
5) is a dynamic array and therefore by definition a resizable array in pascal.

Yes I expressed myself badly again. I had understood and accepted this. My point 5) was about not aiming at making use of SetLength on arrays passed as parameters because different parts of the program or a different program calling an external routine are in charge of sizing.

Omitting range checks in very specific cases for very short parts of a routine to avoid overhead and to enhance efficiency should be safe enough. I will not go the C-do. (I always deemed C ugly when I had to try to understand a piece of code. My original training more than 50 years ago was with ALGOL60 - very strictly typed.)

Thaddy

  • Hero Member
  • *****
  • Posts: 3697
Re: How to declare a variable absolute with respect to a dynamic array?
« Reply #13 on: February 18, 2017, 09:48:35 pm »
Well, you over-stressed my humble experience since 1978....(Bugger.. >:D:) :) :)
But if I can be of any service to olderbies I am glad to help out when possible. O:-)
Maybe I should add my mother is 86 and is perfectly capable to write Delphi database applications (true!)
She didn't switch to Lazarus yet,... Oh, give her time... I am very proud about that fact...and my mother of course,
« Last Edit: February 18, 2017, 10:05:23 pm by Thaddy »
Why do the Danish always try to fuck up any programming language?

Blestan

  • Sr. Member
  • ****
  • Posts: 327
Re: How to declare a variable absolute with respect to a dynamic array?
« Reply #14 on: February 19, 2017, 01:03:28 am »
hahahahaha . speaking of parents .. my father was (born in 1943 and he passed away in 2013) the first programers in bulgaria ever .. programed in algol in the early 60 .. then fortran in the '70 .... cics/cobol  for ibm 360 in the 80'then i theached him to delphi in the '90 .... :)))
thaddy is one of the best agents in this forum ... his comments aways acted for me like a kick in the face but very provacative/productive :)))\
cheers :)))
p.s if you do not take in consideration thaddy's comment you are a complete idiot / morron / nerd ... believe me :))
« Last Edit: February 19, 2017, 01:06:25 am by Blestan »

 

Recent

Get Lazarus at SourceForge.net. Fast, secure and Free Open Source software downloads Open Hub project report for Lazarus