Possible Bug (Mac M1/AArch)?

[Grahame Grieve]

Consider this code:

Code: Pascal  [Select][+][-]

1. procedure foo(s : TStream);
2. var
3.   b : TBytes;
4. begin
5.   SetLength(b, 4000);
6.   s.read(b, 4000);
7. end;
8.  

My code has plenty of examples of this, and I believe it has always worked for me before (Delphi, and FPC windows, unix, and Mac). But on my current Mac m1 with Lazarus 2.2.5 (rev lazarus_2_2_4-3-g564e1e8244) FPC 3.2.2 aarch64-darwin-cocoa, that results in memory corruption and complete failure of the executable thereafter. If I change it to

Code: Pascal  [Select][+][-]

1. procedure foo(s : TStream);
2. var
3.   b : TBytes;
4. begin
5.   SetLength(b, 4000);
6.   s.read(b[0], 4000);
7. end;

that works fine. This feels like a compiler bug to me, should I report it? You can see real world fixes here: https://github.com/HealthIntersections/fhirserver/pull/190/files

[Leledumbo]

I believe it should not instead. TStream.Read expects untyped parameter, which is treated differently based on actual TStream descendant used. But the semantic should be similar to pure array of bytes, no metadata whatsoever to skip. Therefore, b and b[0] are totally different. The former will point to the start of the TBytes structure, which contains metadata (length, refcount, etc.) while the latter is the start of the data part only, which is what you want.

[domasz]

Code: Pascal  [Select][+][-]

1. procedure foo(s : TStream);
2. var
3.   b : TBytes;
4. begin
5.   SetLength(b, 4000);
6.   s.read(b[0], 4000);
7. end;

I've always (over 10 years) used this way in Delphi. This is the only one that makes sense to me.
Also you can read to strings:

Code: Pascal  [Select][+][-]

1. b: AnsiString;
2. [...]
3. S.Read(b[1], 4000);

[ASerge]

I've always (over 10 years) used this way in Delphi. This is the only one that makes sense to me.

Delphi has the method

Code: Pascal  [Select][+][-]

1. TStream.function Read(var Buffer: TBytes; Count: Longint): Longint;

But FPC does not have it.

[PascalDragon]

I've always (over 10 years) used this way in Delphi. This is the only one that makes sense to me.

Delphi has the method

Code: Pascal  [Select][+][-]

1. TStream.function Read(var Buffer: TBytes; Count: Longint): Longint;

But FPC does not have it.

FPC 3.3.1 has it as well. For 3.2.2 you could introduce a class helper that adds that method after all it only does a Read(Buffer[0], Count) in the end (though yes, it goes through the method that provides the Offset parameter, but that doesn't change the principle).

[Grahame Grieve]

I don't see how this from TBytesStream is different to my first code:

Code: Pascal  [Select][+][-]

1. function TBytesStream.Realloc(var NewCapacity: PtrInt): Pointer;
2. begin
3.   // adapt TMemoryStream code to use with dynamic array
4.   if NewCapacity<0 Then
5.     NewCapacity:=0
6.   else
7.     begin
8.       if (NewCapacity>Capacity) and (NewCapacity < (5*Capacity) div 4) then
9.         NewCapacity := (5*Capacity) div 4;
10.       NewCapacity := (NewCapacity + (TMSGrow-1)) and not (TMSGROW-1);
11.     end;
12.   if NewCapacity=Capacity then
13.     Result:=Pointer(FBytes)
14.   else
15.     begin
16.       SetLength(FBytes,Newcapacity);
17.       Result:=Pointer(FBytes);
18.       if (Result=nil) and (Newcapacity>0) then
19.         raise EStreamError.Create(SMemoryStreamError);
20.     end;
21. end;          

[ASerge]

I don't see how this from TBytesStream is different to my first code:

This code does not use the Read method, in which an error occurs when parameters are passed incorrectly.

[Grahame Grieve]

no but it uses the pointer to the TBytes directly, rather than a pointer to TBytes[0]. I don't see how that's different?

[Jonas Maebe]

A dynamic array is internally a pointer. TBytesStream.Realloc changes that pointer to point to a different block of memory of larger/smaller size. That is the purpose of a pointer, so this code is correct.

Your code was reading array data and storing it in the pointer itself, spilling out in memory coming after that pointer. That is wrong, since the read data should be stored in the memory block to which the pointer points (and the pointer just contains the address of that block). That memory can be addressed using tbytesvar[0].

[PascalDragon]

You can use the following class helper with the FPC 3.2.x series to achieve the same that's possible in FPC 3.3.1 now:

Code: Pascal  [Select][+][-]

1. // interface section
2. {$ifdef ver3_2}
3. type
4.   TStreamHelper = class helper for TStream
5.   public
6.     function Read(var Buffer: TBytes; Count: Longint): Longint; overload;
7.     function Read(Buffer: TBytes; aOffset, Count: Longint): Longint; overload;
8.     function Write(const Buffer: TBytes; Offset, Count: Longint): Longint; overload;
9.     function Write(const Buffer: TBytes; Count: Longint): Longint; overload;
10.   end;
11. {$endif}
12.  
13. // implementation section
14. {$ifdef ver3_2}
15. function TStreamHelper.Read(var Buffer: TBytes; Count: Longint): Longint;
16. begin
17.   Result := Read(Buffer, 0, Count);
18. end;
19.  
20. function TStreamHelper.Read(Buffer: TBytes; aOffset, Count: Longint): Longint;
21. begin
22.   Result := Read(Buffer[aOffset], Count);
23. end;
24.  
25. function TStreamHelper.Write(const Buffer: TBytes; Offset, Count: Longint): Longint;
26. begin
27.   Result := Write(Buffer[Offset], Count);
28. end;
29.  
30. function TStreamHelper.Write(const Buffer: TBytes; Count: Longint): Longint;
31. begin
32.   Result := Write(Buffer, 0, Count);
33. end;
34. {$endif}
35.  

Just make sure that the unit is in scope when you want to use the added methods.

[Arioch]

Code: Pascal  [Select][+][-]

1. procedure foo(s : TStream);
2.   s.read(b[0], 4000);
3. end;

I've always (over 10 years) used this way in Delphi

Call me paranoid, but i wrote

Code: Pascal  [Select][+][-]

1.   s.read(b[Low(b)], 4000);

Experience with moving legacy projects to Unicode Delphi and new datatypes (like gen-dyn-arrays instead of raw pointers) and how all hell was breaking loose with the undocumented and subconscious programming against implementation details suddenly were blessing me as unreproduccible shroedin-bugs in the twisted code whose design logic no one remembered any more.