[SOLVED] Passing static array of byte by reference... without creating copies

[BigChimp]

I've read
http://stackoverflow.com/questions/3911432/pass-static-byte-array-in-parameter-in-delphi
and tried to apply it.... without success... or perhaps with other complications.

More eyes welcome

The program recurses through files/directories, and stores details in a Firebird (embedded0 database. Sort of like locate

My goal: have 1 buffer that is used for all md5 calculations instead of having duplicates.
The application is single threaded so 1 buffer should be ok..

In search.pas, the search happens in AddFileToResult
FFileBuffer should create a new array once...

Code: [Select]

  TFileSearch = class
  private
...
    // Buffer that can be used by e.g. md5 hashes
    FFileBuffer: TFileBuffer; //Buffer used e.g. in md5 calculations, defined in flocatecommon as array[1...1024*1024] of byte
...and this is passed to the directoryentry constructor (should be as reference)...

Code: [Select]

procedure TFileSearch.AddFileToResult(const ThisDir: String; item: TSearchRec);
// Finds out properties of file specified in item, adds this information
// to a record, and adds a pointer in the Res list to this record.
var
  DirEntry: TDirectoryEntry;
begin
  assert(ThisDir <> '', 'Directory to be searched may not be empty');
  assert(Item.Name <> '', 'File to be searched must have a filename');
  //Create a new file object and let it figure out all properties of the file.
  DirEntry := TDirectoryEntry.Create(ThisDir + Item.Name,FFileBuffer,FReadFileContents);
... then directoryentry.pas does this

Code: [Select]

  TDirectoryEntry = class(TObject)
  private
...
    FFileBuffer: TFileBuffer; //buffer for e.g. md5 etc can be reused.
... this worries me, AFAIU, this will generate a new array. How can I declare the right variable here?

The constructor then goes on

Code: [Select]

constructor TDirectoryEntry.Create(FullPath: string; var FileBuffer: TFileBuffer; ReadFileContents: boolean);
begin
...
  FFileBuffer:=FileBuffer;

Attached complete code for Win/Linux; it creates a firebird embedded db if it doesn't exist (requires fb dll/lib of course)

Thanks!

[User137]

The format on dynamic and static array are little different, so it's not directly possible i'd assume, without hacks. Easy solution would be to use array pointer

Code: [Select]

type
  PFileBuffer = ^TFileBuffer;
...
private
...
    FFileBuffer: PFileBuffer;

[Leledumbo]

Do as User137 suggests. Even if the parameter is declared by reference, assignment between static arrays does a deep copy. Well actually, even if you don't assign anything to TDirectoryEntry.FFileBuffer, when you create TDirectoryEntry instance, each will already occupy the its own 1 MB space. Assignment only copies the values.

[BigChimp]

Thanks a lot, that does seem to work fine!

[User137]

Another solution came to mind, that FileBuffer would be a array property, which uses getter function directly to original static array. It has benefit of having simpler syntax when coding with, no pointer confusions with ^ symbols.

[BigChimp]

Ok; however I have to pass on the buffer to an MD5 function anyway (see below), so the pointers actually suit me fine...

@Leledumbo: the md5 function is this one:

Code: [Select]

// Adapted from MDFile in md5 unit
function CustomMD5File(const Filename: String; var Buffer: TFileBuffer): TMDDigest;
const
  Version=MD_VERSION_5; //calculate md5 format hash
var
  F: File;
  Context: TMDContext;
  Count: Cardinal;
  OldFileMode: Longint;
begin
  MDInit(Context, Version);

  Assign(F, Filename);
  {$push}{$i-}
  OldFileMode := FileMode;
  FileMode := 0;
  Reset(F, 1);
  {$pop}

  try
    if IOResult = 0 then
    begin
      repeat
        BlockRead(F, Buffer, Bufsize, Count);
        if Count > 0 then
          MDUpdate(Context, Buffer, Count);
      until Count < BufSize;
      Close(F);
    end;

    MDFinal(Context, Result);
  finally
    FileMode := OldFileMode;
  end;
end;
and you said:

Even if the parameter is declared by reference, assignment between static arrays does a deep copy.

Do I understand correctly that I should have memory leaks now? -gh/heaptrace doesn't show any memleaks to do with direntry... so perhaps I misunderstand.

[BigChimp]

Oh, source via
https://bitbucket.org/reiniero/flocate
...

[Leledumbo]

Even if the parameter is declared by reference, assignment between static arrays does a deep copy.

Do I understand correctly that I should have memory leaks now? -gh/heaptrace doesn't show any memleaks to do with direntry... so perhaps I misunderstand.

No, static arrays are allocated on the stack. So it should never introduce memory leak.

[BigChimp]

Thanks, Leledumbo. Couldn't really (quickly) find that documented but well...

[BigChimp]

Found that the updated documentation
http://www.freepascal.org/docs-html/ref/refsu18.html#x42-460003.3.1
specifies "When static array-type variables are assigned to each other, the contents of the whole array is copied." Ok.