Recent

Author Topic: record fields alignment  (Read 712 times)

440bx

  • Hero Member
  • *****
  • Posts: 5880
record fields alignment
« on: November 16, 2025, 08:09:27 pm »
Hello,

The FPC documentation leads the reader to believe that it is possible to tell the compiler to align certain record fields on particular boundaries.   I've tried everything I could imagine and never managed to make it work.

I figured I'd ask for the solution to a specific problem. Hopefully someone knows how to tell the compiler to align fields as necessary.

Consider the following structure definition:
Code: Pascal  [Select][+][-]
  1. type
  2.   { _ALPC_COMPLETION_LIST_HEADER                      128 alignment required  }
  3.  
  4.   { https://nodocpage                                                         }
  5.   { hfile://ntlpcapi.h systeminformer                                         }
  6.  
  7.   { https://www.vergiliusproject.com/kernels/x86/windows-10/22h2/_ALPC_COMPLETION_LIST_HEADER }
  8.   { https://www.vergiliusproject.com/kernels/x64/windows-11/25h2/_ALPC_COMPLETION_LIST_HEADER }
  9.  
  10.   PALPC_COMPLETION_LIST_HEADER = ^TALPC_COMPLETION_LIST_HEADER;
  11.   TALPC_COMPLETION_LIST_HEADER = record
  12.     StartMagic                     : qword;          {   8                    }
  13.  
  14.     TotalSize                      : DWORD;          {   4                    }
  15.     ListOffset                     : DWORD;          {   4                    }
  16.     ListSize                       : DWORD;          {   4                    }
  17.     BitmapOffset                   : DWORD;          {   4                    }
  18.     BitmapSize                     : DWORD;          {   4                    }
  19.     DataOffset                     : DWORD;          {   4                    }
  20.     DataSize                       : DWORD;          {   4                    }
  21.     AttributeFlags                 : DWORD;          {   4                    }
  22.     AttributeSize                  : DWORD;          {   4                    }
  23.  
  24.                                                      {  44 total              }
  25.  
  26.     { field to force alignment                                                }
  27.  
  28.     _ForceAlign128A                : packed array[0..pred(20)] of byte;
  29.     { force next offset to be $40  }                 {  64 total              }
  30.  
  31.     { next field must be 128 bit aligned                                       }
  32.  
  33.     State                          : TALPC_COMPLETION_LIST_STATE;
  34.                                                      {   8                    }
  35.  
  36.     LastMessageId                  : DWORD;          {   4                    }
  37.     LastCallbackId                 : DWORD;          {   4                    }
  38.  
  39.                                                      {  16 subtotal           }
  40.     { field to force alignment                                                }
  41.  
  42.     _ForceAlign128B                : packed array[0..pred(48)] of byte;
  43.     { force next offset to be $80  }                 { 128 total              }
  44.  
  45.     { next 4 fields must be 128 bit aligned                                    }
  46.  
  47.     { 1 - 128 } PostCount          : DWORD;          {   4                    }
  48.  
  49.     { field to force alignment                                                }
  50.  
  51.     _ForceAlign128C                : packed array[0..pred(60)] of byte;
  52.     { force next offset to be $c0  }                 { 192 total              }
  53.  
  54.  
  55.     { 2 - 128 } ReturnCount        : DWORD;          {   4                    }
  56.  
  57.     { field to force alignment                                                }
  58.  
  59.     _ForceAlign128D                : packed array[0..pred(60)] of byte;
  60.     { force next offset to be $100 }                 { 256 total              }
  61.  
  62.  
  63.     { 3 - 128 } LogSequenceNumber  : DWORD;          {   4                    }
  64.  
  65.     { field to force alignment                                                }
  66.  
  67.     _ForceAlign128E                : packed array[0..pred(60)] of byte;
  68.     { force next offset to be $140 }                 { 320 total              }
  69.  
  70.     { 4 - 128 } UserLock           : TRTL_SRWLOCK;
  71.  
  72.  
  73.     EndMagic                       : qword;          { simple qword alignment }
  74.                                                      { 336 total              }
  75.  
  76.     { force the structure size to be a multiple of 128                        }
  77.  
  78.     _ForceSize                     : packed array[0..pred(48)] of byte;
  79.   end;                                                 { 384 total            }
  80.  
  81.   {$if sizeof(TALPC_COMPLETION_LIST_HEADER) <> $180}
  82.     {$FATAL sizeof TALPC_COMPLETION_LIST_HEADER does not equal $180}
  83.   {$endif}
  84.  
The fields State, PostCount, ReturnCount, LogSequenceNumber and UserLock must reside on a 128 bit boundary. 

In MS C, prefixing those fields with DECLSPEC_ALIGN(128) is all that's needed to tell the compiler to align the fields at that boundary. 

Does FPC have something that does what DECLSPEC_ALIGN(128) does ?

The above definition uses explicit "filler" in the form of fields named "_ForceAlignXXX" to get the desired alignment.  This is very cumbersome and error prone (not to mention time consuming to get it right.)  It would be really great if there was a way to tell FPC to align those fields without having to resort to this pedestrian method.

That's the question, is there a simple, straightforward way to tell FPC to align those fields ?

Thank you for your help.
FPC v3.2.2 and Lazarus v4.0rc3 on Windows 7 SP1 64bit.

MathMan

  • Sr. Member
  • ****
  • Posts: 444
Re: record fields alignment
« Reply #1 on: November 16, 2025, 10:29:15 pm »
Hello 440bx,

Initially I thought about subtyping and then using {$packrecords}. But the fundamental issue here is that {$packrecords} only supports sizes 1, 2, 4, 8, 16 and 32.

Sorry, but atm I don't see a simple solution because of this. One could collect fields within the same 128 byte region into a separate structure. Then define your target structure like (top of my head, not tested)

Code: Pascal  [Select][+][-]
  1. PART_ONE = record
  2.     StartMagic                     : qword;          {   8                    }
  3.  
  4.     TotalSize                      : DWORD;          {   4                    }
  5.     ListOffset                     : DWORD;          {   4                    }
  6.     ListSize                       : DWORD;          {   4                    }
  7.     BitmapOffset                   : DWORD;          {   4                    }
  8.     BitmapSize                     : DWORD;          {   4                    }
  9.     DataOffset                     : DWORD;          {   4                    }
  10.     DataSize                       : DWORD;          {   4                    }
  11.     AttributeFlags                 : DWORD;          {   4                    }
  12.     AttributeSize                  : DWORD;          {   4                    }
  13. end;
  14.  
  15. TALPC_COMPLETION_LIST_HEADER = record
  16.   A: PART_ONE;
  17.   FILL_A: array[ 0..Pred[ SizeOf( PART_ONE )] of UInt8;
  18. ...
  19. end;
  20.  

But this would only take out potential size errors in the definition of the fillers - and add additional typing due to one more level of indirection. And as I know that you want to stay as close as possible to original code designs it's probably no real solution.

creaothceann

  • Full Member
  • ***
  • Posts: 220
Re: record fields alignment
« Reply #2 on: November 16, 2025, 10:51:17 pm »
You can define the fields as properties:

Code: Pascal  [Select][+][-]
  1. program Test_Alignment;  {$ModeSwitch AdvancedRecords}
  2. uses
  3.         SysUtils;
  4.  
  5. type
  6.         u8   = System  . Byte;
  7.         u16  = System  . Word;
  8.         u32  = System  .DWord;
  9.         u64  = System  .QWord;
  10.         u128 = SysUtils.Int128Rec;
  11.  
  12. const
  13.         //...
  14.         Bit24 = u32(1 SHL 24);    Bits24 = u32(Bit24 - 1);
  15.         //...
  16.  
  17. type
  18.         //...
  19.         u24 = 0..Bits24;
  20.         //...
  21.  
  22.  
  23.         TALPC_Completion_List_State = bitpacked record  // https://www.vergiliusproject.com/kernels/x64/windows-11/25h2/_ALPC_COMPLETION_LIST_STATE
  24.                 case u8 of
  25.                         0: (Head, Tail : u24;  ActiveThreadCount : u16);
  26.                         1: (Value                                : u64);
  27.                 end;
  28.  
  29.  
  30.         TRTL_SRWLOCK = record  // https://www.pilotlogic.com/codetyphon/webhelp/pl_win_api/pl_win_api/win.synchapi/trtl_srwlock.html
  31.                 Ptr : pointer;
  32.                 end;
  33.  
  34.  
  35.         // _ALPC_COMPLETION_LIST_HEADER    128 alignment required
  36.         // https://nodocpage
  37.         // hfile://ntlpcapi.h systeminformer
  38.         // https://www.vergiliusproject.com/kernels/x86/windows-10/22h2/_ALPC_COMPLETION_LIST_HEADER
  39.         // https://www.vergiliusproject.com/kernels/x64/windows-11/25h2/_ALPC_COMPLETION_LIST_HEADER
  40.  
  41.         pALPC_Completion_List_Header = ^TALPC_Completion_List_Header;
  42.  
  43.         TALPC_Completion_List_Header = packed record
  44.                 StartMagic        : u64;                                //   0  8
  45.                 TotalSize         : u32;                                //   8  4
  46.                 ListOffset        : u32;                                //  12  4
  47.                 ListSize          : u32;                                //  16  4
  48.                 BitmapOffset      : u32;                                //  20  4
  49.                 BitmapSize        : u32;                                //  24  4
  50.                 DataOffset        : u32;                                //  28  4
  51.                 DataSize          : u32;                                //  32  4
  52.                 AttributeFlags    : u32;                                //  36  4
  53.                 AttributeSize     : u32;                                //  40  4
  54.                 __padding_1__     : packed array[0..pred(20)] of byte;  //  44 20
  55.                 State             : TALPC_COMPLETION_LIST_STATE;        //  64  8
  56.                 LastMessageId     : u32;                                //  72  4
  57.                 LastCallbackId    : u32;                                //  76  4
  58.                 __padding_2__     : packed array[0..pred(48)] of byte;  //  80 48
  59.                 PostCount         : u32;                                // 128  4
  60.                 __padding_3__     : packed array[0..pred(60)] of byte;  // 132 60
  61.                 ReturnCount       : u32;                                // 192  4
  62.                 __padding_4__     : packed array[0..pred(60)] of byte;  // 196 60
  63.                 LogSequenceNumber : u32;                                // 256  4
  64.                 __padding_5__     : packed array[0..pred(60)] of byte;  // 260 60
  65.                 UserLock          : TRTL_SRWLOCK;                       // 320  8
  66.                 EndMagic          : u64;                                // 328  8
  67.                 __padding_6__     : packed array[0..pred(48)] of byte;  // 336 48  // 384 total
  68.                 end;
  69.                 {$if SizeOf(TALPC_COMPLETION_LIST_HEADER) <> 384}
  70.                         {$FATAL sizeof TALPC_COMPLETION_LIST_HEADER does not equal 384}
  71.                 {$endif}
  72.  
  73.  
  74.         TALPC_Completion_List_Header_v2 = record
  75.                 private const _Size       = 384;                         // may be an odd size
  76.                 private const _PaddedSize = ((_Size + 15) div 16) * 16;  // padded to 128 bits
  77.                 private var _ : packed record
  78.                         var
  79.                         case u8 of
  80.                                 0: (_8   : array[0..(_PaddedSize div  1) - 1] of u8  );
  81.                                 1: (_16  : array[0..(_PaddedSize div  2) - 1] of u16 );
  82.                                 2: (_32  : array[0..(_PaddedSize div  4) - 1] of u32 );
  83.                                 3: (_64  : array[0..(_PaddedSize div  8) - 1] of u64 );
  84.                                 4: (_128 : array[0..(_PaddedSize div 16) - 1] of u128);
  85.                         end;
  86.                 public property StartMagic : u64 read _._64[0];
  87.                 //...
  88.                 end;
  89.  
  90.  
  91. begin
  92.         WriteLn(SizeOf(pALPC_Completion_List_Header(NIL)^));
  93.         WriteLn(SizeOf(TALPC_Completion_List_Header_v2   ));
  94.         ReadLn;
  95. end.

This removes the ability to get the address of such a "field" though, and it works only for 1-, 2-, 4- or 8-byte fields.
EDIT: SysUtils has Int128Rec that could be used for 16-byte alignment.
« Last Edit: November 17, 2025, 12:00:58 am by creaothceann »
Quote from: Thaddy
And don't start an argument, I am right.
Quote from: Thaddy
You have a thorough misunderstanding of what I wrote. Can you provide an example this time? I doubt it. (because you never do out of incompentence)

440bx

  • Hero Member
  • *****
  • Posts: 5880
Re: record fields alignment
« Reply #3 on: November 17, 2025, 12:29:45 am »
Thank you MathMan and Creaothceann for the suggestions.

Unfortunately anything that requires manual padding just isn't a solution.

Aligning in FPC is a baroque situation.


FPC v3.2.2 and Lazarus v4.0rc3 on Windows 7 SP1 64bit.

jamie

  • Hero Member
  • *****
  • Posts: 7379
Re: record fields alignment
« Reply #4 on: November 17, 2025, 01:09:25 am »
You don't need to fix the alignment; you have the offsets and that is all you need.

For all the values starting at the first struct, you can use advanced records for each one, on the read and write for
properties.
   
 The compiler will nicely hide the fact of what is going one.

write a setter and getter for each one and mark them inline.
Jamie
The only true wisdom is knowing you know nothing

creaothceann

  • Full Member
  • ***
  • Posts: 220
Re: record fields alignment
« Reply #5 on: November 17, 2025, 01:39:48 am »
you have the offsets [...] write a setter and getter for each one and mark them inline

I think the main idea is that this should happen automatically.

Unless the compiler writers agree to increase the parameters of {$PackRecords}, perhaps a tool could be written that calculates the offsets and generates the Free Pascal code. (For conversions of already-existing C code.)
« Last Edit: November 17, 2025, 01:41:23 am by creaothceann »
Quote from: Thaddy
And don't start an argument, I am right.
Quote from: Thaddy
You have a thorough misunderstanding of what I wrote. Can you provide an example this time? I doubt it. (because you never do out of incompentence)

jamie

  • Hero Member
  • *****
  • Posts: 7379
Re: record fields alignment
« Reply #6 on: November 17, 2025, 01:44:30 am »
Yes, I understand that but, to get things rolling and still have it work out.
Code: Pascal  [Select][+][-]
  1. unit Unit1;
  2.  
  3. {$mode Delphi}{$H+}
  4.  
  5. interface
  6.  
  7. uses
  8.   Classes, SysUtils, Forms, Controls, Graphics, Dialogs, StdCtrls;
  9.  
  10. type
  11.   PALPC_COMPLETION_LIST_HEADER = ^_ALPC_COMPLETION_LIST_HEADER;
  12.  
  13.  { _ALPC_COMPLETION_LIST_HEADER }
  14.  
  15.  _ALPC_COMPLETION_LIST_HEADER = Packed Record
  16.     Private
  17.      Function fGetState:Uint64; Inline;
  18.      Procedure FSetState(aValue:Uint64);Inline;
  19.      Function  GetLastMessageId:DWord;Inline;
  20.      Procedure SetLastMessageID(aValue:DWORD); Inline;
  21.     Public
  22.     StartMagic:Uint64; //0x0
  23.     TotalSize:Dword;  //0x8
  24.     ListOffset:DWord; //0xc
  25.     ListSize:DWord;    //0x10
  26.     BitmapOffset:DWord; //0x14
  27.     BitmapSize:DWord;                                                       //0x18
  28.     DataOffset:Dword;                                                      //0x1c
  29.     DataSize:DWord;                                                         //0x20
  30.     AttributeFlags:Dword;                                                   //0x24
  31.     AttributeSize:Dword;                                                   //0x28;
  32.     Dum:Array[0..$180-SizeOF(_ALPC_COMPLETION_LIST_HEADER)-1] Of Byte;
  33.   Property state:Uint64 Read fGetState write fSetState;                   //0x40;
  34.   Property LastMessageID:Dword Read GetLastMessageID write SetLastMessageId; //0x48
  35.  end;
  36.  
  37.   { TForm1 }
  38.  
  39.   TForm1 = class(TForm)
  40.     Button1: TButton;
  41.     procedure Button1Click(Sender: TObject);
  42.   private
  43.  
  44.   public
  45.  
  46.   end;
  47.  
  48. var
  49.   Form1: TForm1;
  50.  
  51. implementation
  52.  
  53. {$R *.lfm}
  54.  
  55. { _ALPC_COMPLETION_LIST_HEADER }
  56.  
  57. function _ALPC_COMPLETION_LIST_HEADER.fGetState: Uint64;
  58. begin
  59.   Result := PUint64(@Self)[$40];
  60. end;
  61.  
  62. procedure _ALPC_COMPLETION_LIST_HEADER.FSetState(aValue: Uint64);
  63. begin
  64.  PUint64(@Self)[$40]:=aValue;
  65. end;
  66.  
  67. function _ALPC_COMPLETION_LIST_HEADER.GetLastMessageId: DWord;
  68. begin
  69.  Result := PUint64(@Self)[$48];
  70. end;
  71.  
  72. procedure _ALPC_COMPLETION_LIST_HEADER.SetLastMessageID(aValue: DWORD);
  73. begin
  74.  PUint64(@Self)[$48]:=aValue;
  75. end;
  76.  
  77. { TForm1 }
  78.  
  79. procedure TForm1.Button1Click(Sender: TObject);
  80. begin
  81.  caption :=SizeOf(_ALPC_COMPLETION_LIST_HEADER).ToHexString;
  82. end;
  83.  
  84. end.
  85.  
  86.  

I put a test button down at the button to view the size;`

The remaining properties can be filled in.

The only true wisdom is knowing you know nothing

creaothceann

  • Full Member
  • ***
  • Posts: 220
Re: record fields alignment
« Reply #7 on: November 17, 2025, 02:04:06 am »
As I showed with the StartMagic field, you don't need getter/setter subroutines. Just use the offset in the property declaration.
Quote from: Thaddy
And don't start an argument, I am right.
Quote from: Thaddy
You have a thorough misunderstanding of what I wrote. Can you provide an example this time? I doubt it. (because you never do out of incompentence)

jamie

  • Hero Member
  • *****
  • Posts: 7379
Re: record fields alignment
« Reply #8 on: November 17, 2025, 02:20:58 am »
I've used the address from the property before however, I was told by one of the DEV;s of FPC I shouldn't be doing such hacks. Because it's not in the language guide and at the time, I was able to do that with private fields to gain access to them :D

 That was back a couple FPC versions.

Jamie

The only true wisdom is knowing you know nothing

440bx

  • Hero Member
  • *****
  • Posts: 5880
Re: record fields alignment
« Reply #9 on: November 17, 2025, 04:44:09 am »
I think the main idea is that this should happen automatically.
That's exactly right.

In addition to all the problems faced to align fields, the record itself still isn't aligned and requires bending over backwards to have it on a 128 bit boundary and, of course, a padding field is necessary to make the record size a multiple of the required alignment.

In MS C, getting all that stuff done is downright trivial:  DECLSPEC_ALIGN(128), that's it, as simple as it gets and as it should be.

In a few seconds the declaration is done and correct. 
FPC v3.2.2 and Lazarus v4.0rc3 on Windows 7 SP1 64bit.

creaothceann

  • Full Member
  • ***
  • Posts: 220
Re: record fields alignment
« Reply #10 on: November 17, 2025, 10:41:47 am »
requires bending over backwards to have it on a 128 bit boundary

A custom allocator?
Quote from: Thaddy
And don't start an argument, I am right.
Quote from: Thaddy
You have a thorough misunderstanding of what I wrote. Can you provide an example this time? I doubt it. (because you never do out of incompentence)

440bx

  • Hero Member
  • *****
  • Posts: 5880
Re: record fields alignment
« Reply #11 on: November 17, 2025, 10:49:21 am »
A custom allocator?
Yes, and usually the simplest and straightforward way is to use VirtualAlloc which returns a 64K aligned block, that's definitely 128 bit aligned :)

Other ways are more complicated because the "normal" memory management functions are usually totally oblivious to alignment (they usually return something that is either word or dword aligned but, that's not documented and cannot be depended on.)

Alignment is a really simple feature to implement.  It's really unfortunate that FPC does not offer that feature.
FPC v3.2.2 and Lazarus v4.0rc3 on Windows 7 SP1 64bit.

 

TinyPortal © 2005-2018