Infernal error when decompressing GZIP stream

[hedgehog]

Hi!

I've always decompressed compressed streams using the PASZLIB.
But it didn't work for this file.

I tried to use TDecompressionStream... EDecompressionError!

Finally, I tried using the ZFlate library (author: Fibonacci).

In the first case (ZFlate1 button), the function returns the correct result, the size of the decompressed data is 32768 bytes.

Code: Pascal  [Select][+][-]

1. procedure TForm1.btnZFlate1Click(Sender: TObject);
2. var
3.   bs: TBytesStream;
4.   byOut: TBytes;
5. begin
6.     bs:= TBytesStream.Create;
7.     bs.LoadFromFile('myfile1.gz');
8.     byOut:= zdecompress(bs.Bytes);
9.     Memo1.Lines.Add('Size of uncompressed data: '+IntToStr(Length(byOut))); // 32768 <-- OK!
10.     bs.Free;
11. end;

I noticed something strange here:

Code: Pascal  [Select][+][-]

1. bs.size =  20086
2. Length(bs.Bytes = 20480

Shouldn't the sizes match?

In the second case (the ZFlate2 button), the function returns an incorrect result, the size of the decompressed data is 65536 bytes. That is, double the data!

Code: Pascal  [Select][+][-]

1. procedure TForm1.btnZFlate2Click(Sender: TObject);
2. var
3.   ms: TMemoryStream;
4.   byIn, byOut: TBytes;
5. begin
6.     ms:= TMemoryStream.Create;
7.     ms.LoadFromFile('myfile1.gz');
8.     SetLength(byIn, ms.Size);
9.     ms.ReadBuffer(byIn[0], ms.Size);
10.     byOut:= zdecompress(byIn);
11.     Memo1.Lines.Add('Size of uncompressed data: '+IntToStr(Length(byOut))); // 65536 <-- VERY BAD !
12.     bs.Free;
13. end;

What am I doing wrong?
Is there a problem with the compressed data?

I am attaching a test project.
(Window 10 64 bit + Laz 4.4 64 bit)

[440bx]

"infernal error", that must be an error from hell.  I didn't know FPC could detect those.

[Fibonacci]

This is not .gz (GZIP) - it's ZLIB format, and the file is corrupted (invalid checksum). Since the file is broken, ZFlate "assumed" it should still try to inflate the data, which worked - but that doesn't change the fact that the file isn't GZIP and is corrupted.

[hedgehog]

Hi, Fibonacci
I came up with an arbitrary file extension. This is a data stream from industrial equipment.
What confuses me is that in the second case, the result is doubled.

I also noticed that if you add an extra byte to the array of bytes, everything works fine.

Code: Pascal  [Select][+][-]

1. procedure TForm1.btnZFlate2Click(Sender: TObject);
2. var
3.   ms: TMemoryStream;
4.   byIn, byOut: TBytes;
5. begin
6.     ms:= TMemoryStream.Create;
7.     ms.LoadFromFile('myfile1.gz');
8.     SetLength(byIn, ms.Size + 1); // <<------------------ this
9.     ms.ReadBuffer(byIn[0], ms.Size);
10.     byOut:= zdecompress(byIn);
11.     Memo1.Lines.Add('Size of uncompressed data: '+IntToStr(Length(byOut))); //
12.     bs.Free;
13. end;

[ginoo]

The new generation of fpc errors, the infernal errors, the most terrible, the most frightening