I think I have understood what is going on.
I found that if I put a WriteLn(Result.SomeString); at the top of my GetRec() function like so:
function GetRec(L: String): TSomeRec;
begin
WriteLn('Result: ' + Result.SomeString);
FillByte(Result{%H-}, SizeOf(Result), 0); // without this no memory leak!
Result.SomeString := L;
end;
then it will give the following output:
Result:
Result: str number 0
Result: str number 1
Result: str number 2
Result: str number 3
Result: str number 4
Result: str number 5
Result: str number 6
Result: str number 7
Result: str number 8
Result: str number 9
which leads me to the assumption that the compiler will treat this function call as if Result was passed as a var argument:
procedure GetRec(L: String; var Result: TSomeRec);
begin
FillByte(Result{%H-}, SizeOf(Result), 0); // sabotage FPC reference counting!
Result.SomeString := L;
end;
and call it like so:
for S in SL do
begin
GetRec(S, R);
end;
This will produce the exact same symptoms. So what happens is I erroneously assumed I get a brand new fresh result variable which which I am free to do what I want before I return it, that it is created and owned by the function that returns it but in reality the
caller of the function will provide the result variable and in this case it still contains a valid string from the previous iteration.
Result contains a valid string with a valid reference count, it is not some random garbage. I MUST use the proper fpc way of assigning a new string, only this will ensure it will decref the previously contained string.
FillByte would wipe out the pointer to a perfectly valid string that still has reference count without first dealing with the refcount and disposing it if not used anymore.
The problem becomes much clearer when looking at the problematic code when written like so:
procedure GetRec(L: String; var Result: TSomeRec);
begin
// there is a valid string in here already, we cannot just wipe it with FillByte
// because if we do so FPC has no chance to properly decref the old string.
FillByte(Result{%H-}, SizeOf(Result), 0); // leaves a string dangling!
Result.SomeString := L;
end;
var
SL: TStringList;
R: TSomeRec;
I: Integer;
S: String;
begin
SL := TStringList.Create;
for I := 0 to 10 do
begin
SL.Add(Format('str number %d', [I]));
end;
for S in SL do
begin
GetRec(S, R);
end;
SL.Free;
end.
In the first iteration we assign a string to R.SomeString. In the second and all following iterations we null the pointer with methods outside of fpc's control, thereby leave the memory of the previous string dangling and assign a new string to the record field.
Finalize() or a proper assignment (not raw byte copying) will take care of the previous contents of the result variable and orderly decref and dispose any valid strings that might still be in there.
Lessons learned:
- The result variable might contain valid managed types that originate from somewhere inside the caller of the function or from previous calls in a loop, never null it with FillByte()
- Since 3.0.0 there is a compiler intrinsic Default() that is much more elegant for zeroing records than the usage of FillByte(). Using this will involve the assignment operator instead of raw byte copying and therefore take care of all the reference counting needs when overwriting the contents of managed types.