Recent

Author Topic: Zero Sized Array at end of Records, can this test code be verified ?  (Read 16090 times)

jamie

  • Hero Member
  • *****
  • Posts: 6090
I use code in a couple of projects but I am using 3.0.4 for various reasons.

I ported over a project to the trunk install I have that still uses 3.2.0 and unless I have issues else where I am not so sure its working as it does with 3.0.4 or I have other issues related to trunk I am not seeing atm.
Below is a simple Laz project but can be done in console mode for those that like that better.
I also attached the project.

I just need to know if the values $FFFFFFFF, 1 show on output with higher than 3.0.4 fpc and if you see any issues there.

This all works fine with delphi, it's an old project.
Code: Pascal  [Select][+][-]
  1. unit Unit1;
  2.  
  3. {$mode objfpc}{$H+}
  4. {$modeSwitch advancedRecords}
  5.  
  6. interface
  7.  
  8. uses
  9.   Classes, SysUtils, Forms, Controls, Graphics, Dialogs, StdCtrls;
  10.  
  11. type
  12.   TMyEmptyIntArray= Packed Record
  13.     Function fGetInteger(Index:Integer):Integer; inline;
  14.     procedure fSetInteger(Index:Integer;Avalue:Integer); Inline;
  15.     property Items[index:integer]:Integer read fgetInteger write fsetInteger; default;
  16.   end;
  17.   TMyIntBaseRecord = Packed Record
  18.     IntCount:DWORD;
  19.     Ints:TMyEmptyIntArray; {an Empty field at the end}
  20.   end;
  21.    PMyIntBaseRecord = ^TMyIntBaseRecord;
  22.   { TForm1 }
  23.  
  24.   TForm1 = class(TForm)
  25.     Button1: TButton;
  26.     procedure Button1Click(Sender: TObject);
  27.   private
  28.  
  29.   public
  30.  
  31.   end;
  32.  
  33. var
  34.   Form1: TForm1;
  35.  
  36. implementation
  37.  
  38. {$R *.lfm}
  39. Function TmyEmptyIntArray.fGetInteger(Index:Integer):Integer; inline;
  40. begin
  41.   Result :=PInteger(@Self)[Index];
  42. end;
  43. Procedure TMyEmptyIntArray.fSetInteger(Index:Integer;AValue:Integer);Inline;
  44. Begin
  45.  Pinteger(@Self)[Index]:=aValue;
  46. end;
  47.  
  48. { TForm1 }
  49.  
  50. procedure TForm1.Button1Click(Sender: TObject);
  51. Var
  52.   T:PMyIntBaseRecord;
  53. begin
  54.   T := GetMem(SizeOf(T)+SizeOf(integer)); {Create an item plus one integer at end}
  55.   T^.IntCount := $FFFFFFFF;
  56.   T^.Ints[0]:= 1;
  57.   Caption := T^.IntCount.ToHexString+','+T^.ints[0].Tostring;
  58.   FreeMem(T);
  59. end;
  60.  
  61. end.
  62.  
  63.  

This is just a test app but it represents in small part of a much larger app.

The only true wisdom is knowing you know nothing

MarkMLl

  • Hero Member
  • *****
  • Posts: 6676
Re: Zero Sized Array at end of Records, can this test code be verified ?
« Reply #1 on: December 02, 2021, 05:42:38 pm »
Debian Linux "Buster" 64-bit Lazarus 2.0.12 FPC 3.2.0 shows FFFFFFFF,1 on the title bar after the button is pressed.

HTH, MarkMLl
MT+86 & Turbo Pascal v1 on CCP/M-86, multitasking with LAN & graphics in 128Kb.
Pet hate: people who boast about the size and sophistication of their computer.
GitHub repositories: https://github.com/MarkMLl?tab=repositories

dseligo

  • Hero Member
  • *****
  • Posts: 1196
Re: Zero Sized Array at end of Records, can this test code be verified ?
« Reply #2 on: December 03, 2021, 12:21:47 am »
FFFFFFFF,1 on Lazarus 2.0.10, FPC 3.2.0, Windows 11

trev

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 2020
  • Former Delphi 1-7, 10.2 user
Re: Zero Sized Array at end of Records, can this test code be verified ?
« Reply #3 on: December 03, 2021, 12:40:21 am »
FFFFFFFF,1 on Lazarus 2.3.0 (git main), FPC 3.3.1 (git main), macOS 12.0.1 (aarch64).

marcov

  • Administrator
  • Hero Member
  • *
  • Posts: 11383
  • FPC developer.
Re: Zero Sized Array at end of Records, can this test code be verified ?
« Reply #4 on: December 03, 2021, 10:06:53 am »
It is a quite neat trick if it is legal. (that self of a nested record points to the nested record, not the encompassing one)

Zvoni

  • Hero Member
  • *****
  • Posts: 2319
Re: Zero Sized Array at end of Records, can this test code be verified ?
« Reply #5 on: December 03, 2021, 10:14:11 am »
FFFFFFFF,1 on Laz 2.0.12/FPC3.2.0 32 Bit on Win10 Pro 64 Bit
One System to rule them all, One Code to find them,
One IDE to bring them all, and to the Framework bind them,
in the Land of Redmond, where the Windows lie
---------------------------------------------------------------------
Code is like a joke: If you have to explain it, it's bad

howardpc

  • Hero Member
  • *****
  • Posts: 4144
Re: Zero Sized Array at end of Records, can this test code be verified ?
« Reply #6 on: December 03, 2021, 12:29:46 pm »
It is a quite neat trick if it is legal. (that self of a nested record points to the nested record, not the encompassing one)
Although how could a nested Self point to anything else? It has no knowledge of being nested, it knows only itself.

jamie

  • Hero Member
  • *****
  • Posts: 6090
Re: Zero Sized Array at end of Records, can this test code be verified ?
« Reply #7 on: December 03, 2021, 12:40:29 pm »
It is a quite neat trick if it is legal. (that self of a nested record points to the nested record, not the encompassing one)

 I would hope it would be legal, I've doing such tricks for some time now and the SELF. It would be broken if not pointing to the immediate record when referenced which btw is also part of the parent record that over hangs at the end. That is the intent here.

 This is to support Structs (Records) with empty count arrays at the end of the struct(record).


 The test app I posted here is just a model of a much larger app with a lot more content in the records of course and I did last night move it to the PC with the Trunk install and that seems to work but the app as a whole appears to be getting bad data when using 3.2.x and I need to determine where this is taking place so I can move on.

BTW, I discovered that the OBJECT model still supports this and can be used that way too instead of an advanced record, meaning the tail record can be an object instead.
 The old original code was this way to support the over hang at the end.

 Thank you much for testing, all of you.

The only true wisdom is knowing you know nothing

Warfley

  • Hero Member
  • *****
  • Posts: 1499
Re: Zero Sized Array at end of Records, can this test code be verified ?
« Reply #8 on: December 03, 2021, 01:34:51 pm »
In C code you can often see people use a similar trick:
Code: C  [Select][+][-]
  1. struct {
  2.   ...
  3.   int items[0];
  4. }
This also works in Pascal:
Code: Pascal  [Select][+][-]
  1.   TTestRec = record
  2.     A: Integer;
  3.     Items: Array[0..0] of Integer;
  4.   end;
But throws warnings


Note: you should also not forget, if using that, to always call Initialize and Finalize on the memory for managed types

marcov

  • Administrator
  • Hero Member
  • *
  • Posts: 11383
  • FPC developer.
Re: Zero Sized Array at end of Records, can this test code be verified ?
« Reply #9 on: December 03, 2021, 01:39:49 pm »
Warfley: the pascal definition has one element and thus enlarges the record

Warfley

  • Hero Member
  • *****
  • Posts: 1499
Re: Zero Sized Array at end of Records, can this test code be verified ?
« Reply #10 on: December 03, 2021, 01:50:26 pm »
You are right, there is no way in Pascal to make a 0 sized array.

I've found another solution that does not throw warnings and requires less code:
Code: Pascal  [Select][+][-]
  1. type
  2.   PTestRec = ^TTestRec;
  3.   TTestRec = packed record
  4.     __startofitems: packed record end;
  5.     function Items: PInteger; inline;
  6.   end;
  7.  
  8. function TTestRec.Items: PInteger;
  9. begin
  10.   Result := @__startofitems;
  11. end;
  12.  
  13. var
  14.   p: PTestRec;
  15. begin
  16.   WriteLn(SizeOf(TTestRec));
  17.   p := GetMem(SizeOf(TTestRec) + 4 * SizeOf(Integer));
  18.   p^.Items[2] := 42;
  19.   WriteLn(p^.Items[2]);
But compiling this let's the FPC enter an infinite loop with 100% CPU usage so. This is probably a bug in the FPC

Edit: I am kinda an idiot, there is absolutely no need for that dummy field (which is kinda funny because I used this exact mechanism already multiple times in my code, and never used a dummy field before, don't know why I wanted to use it now):
Code: Pascal  [Select][+][-]
  1. type
  2.   PTestRec = ^TTestRec;
  3.   TTestRec = packed record
  4.     function Items: PInteger; inline;
  5.   end;
  6.  
  7. function TTestRec.Items: PInteger;
  8. begin
  9.   Result := PInteger(Pointer(@self) + SizeOf(Self));
  10. end;
  11.  
  12. var
  13.   p: PTestRec;
  14. begin
  15.   WriteLn(SizeOf(TTestRec));
  16.   p := GetMem(SizeOf(TTestRec) + 4 * SizeOf(Integer));
  17.   p^.Items[2] := 42;
  18.   WriteLn(p^.Items[2]);
  19. end.
This now also makes the FPC compile it.
« Last Edit: December 03, 2021, 01:56:03 pm by Warfley »

jamie

  • Hero Member
  • *****
  • Posts: 6090
Re: Zero Sized Array at end of Records, can this test code be verified ?
« Reply #11 on: December 03, 2021, 02:53:16 pm »
Interesting you found a compiler loop bug, maybe that should be reported ?

In any case, if you were to have a record with the last valid field being the same type as the empty array you can pull this trick which also works.

Code: Pascal  [Select][+][-]
  1. TTheRecord = Record
  2.  Case Byte of
  3.    Count:(CountInt:Integer);
  4.    Ints:Array[-1..-1] of Integer;
  5. End;  
  6.  
;
 Just  index it starting at 0

BUt this  of course only works if the  prior data type is the same

The only true wisdom is knowing you know nothing

marcov

  • Administrator
  • Hero Member
  • *
  • Posts: 11383
  • FPC developer.
Re: Zero Sized Array at end of Records, can this test code be verified ?
« Reply #12 on: December 03, 2021, 03:06:42 pm »
I have some doubts about the sizeof(self)  (always pointer size =4/8 or nested record size=0?)

Warfley

  • Hero Member
  • *****
  • Posts: 1499
Re: Zero Sized Array at end of Records, can this test code be verified ?
« Reply #13 on: December 03, 2021, 03:10:39 pm »
Quote
In any case, if you were to have a record with the last valid field being the same type as the empty array you can pull this trick which also works.
This only works for non managed types. Managed types like strings can not occur in the variant parts of a record. But yes, in this case, it would be possible without requireing a function.

I have some doubts about the sizeof(self)  (always pointer size =4/8 or nested record size=0?)
This is actually quite a good question. The pointer size does not really matter but the record size does. I think that with packed it should always be 0 but I am not sure if this is the case and/or is documented somewhere.

One exception I could think of is, when more advanced RTTI comes, that records might get hidden RTTI fields added if enabled. But TBH I don't know how the plans are for RTTI in advanced records

jamie

  • Hero Member
  • *****
  • Posts: 6090
Re: Zero Sized Array at end of Records, can this test code be verified ?
« Reply #14 on: December 03, 2021, 03:29:50 pm »
Quote
This only works for non managed types. Managed types like strings can not occur in the variant parts of a record. But yes, in this case, it would be possible without requireing a function.


Well I don't use managed types in this type of code since it's reading external data to start with.

I do have array of char in the records and I use Short strings at times.
The only true wisdom is knowing you know nothing

 

TinyPortal © 2005-2018