Forum > General
Indy Gzip library compression problem
dinmil:
Hello
I wrote some http server application for windows 64 /32 which use gzip indy compression to compress output.
Problem is that from time to time I get zlib error -5 when I try to compress some string.
In attachment you can find my isolated test case which proves that some string can not be compressed using Indy routines.
I thing this is bug inside Indy Library, and I need confirmation or fix. Any help is appreciated.
(*
The purpose of test is to show that Indy
library can not GZIP compress some string
String is saved in file uncompresable.xml
String is loaded into memory and
I cut peace of files and compresse it
After some time I found on peace which
cannot be compressed and program returns
zlib error -5 which means that buffer is
not good. If you change MemLevel from 9 to
lower values or compression level from 9 to
no lower values you can compress this string
but eventually if you try some other strings
you get the same result (-5).
Indy use zlib1.dll (ver 1.2.8) for 32 bit windows
and zlibwapi.dll (ver 1.2.5) for 64 bit windows
I think that something is wrong with Indy
implementation of compression and the way
how -5 error should be interpreted but I'm
not sure. I tried some other pascal zlib libraries
and everyhing is good.
Used version is Windows CodeTyphon 5.1.0, Fpc = 2.7.1,
pl_Indy Source (24-10-2014 SVN Rev 5201)
I appreciate any help from Indy team or CodeTyphon team or
Lazarus team
*)
dll._zip contain zlib1 for 32 windows and zlibwapi from 64 bit windows.
Rename and uncompress it if you do not have your own version.
Regards, Dinko
ChrisF:
Apparently, you've got a problem with an incomplete or empty buffer case.
And it seems it's not really a new problem, though not exactly yours. See: http://forums2.atozed.com/viewtopic.php?f=7&t=26206
After just a few tests, the following fix was OK for me (but I'm not an expert): Indy 10.6.1 rev 5217 and zlib version 1.2.8.0
In Lib/Protocols/IdZlib.pas, in procedure IndyCompressStream:
--- Code: ---...
repeat
//&&&& Fix (add the following lines)
if (strm.avail_in = 0) and (strm.avail_out = 0) then begin
WriteOut;
end;
//&&&& Fix end
Finished := CCheck(deflate(strm, Z_FINISH)) = Z_STREAM_END;
WriteOut;
until Finished;
...
--- End code ---
Anyway, I suggest you to report your problem to Remy Lebeau in the Indy forum: http://forums2.atozed.com/index.php,
in order to get a full a more complete answer, and a more proper fix for your problem.
dinmil:
Thanks, for quick response. I will check soon your fix and report results.
dinmil:
After some test I done with your fix implemented, I can confirm that this fix solve the problem for me.
I'm still waiting Indy guys for answers.
I wrote test program which gzip and ungzip many strings and checking that Input and Output strings are the same.
So far they are, program is still running :-)
Thank you. You saved my life :-)
Remy Lebeau:
--- Quote from: ChrisF on December 10, 2014, 03:27:01 pm ---And it seems it's not really a new problem, though not exactly yours. See: http://forums2.atozed.com/viewtopic.php?f=7&t=26206
--- End quote ---
The problem is, that discussion was never finalized, so no changes were made to Indy. I'm not a ZLib expert, and I didn't (still don't) have to debug and test ZLib issues.
--- Quote from: ChrisF on December 10, 2014, 03:27:01 pm ---After just a few tests, the following fix was OK for me (but I'm not an expert): Indy 10.6.1 rev 5217 and zlib version 1.2.8.0
--- End quote ---
Which do you think is better?
--- Code: ---...
repeat
//&&&& Fix (add the following lines)
if (strm.avail_in = 0) and (strm.avail_out = 0) then begin
WriteOut;
end;
//&&&& Fix end
Finished := CCheck(deflate(strm, Z_FINISH)) = Z_STREAM_END;
WriteOut;
until Finished;
...
--- End code ---
Or the fix proposed in the earlier discussion:
--- Code: ---var
CResult: Integer;
{... snip ...}
repeat
CResult := deflate(strm, Z_FINISH);
if CResult <> Z_BUF_ERROR then
begin
CCheck(CResult);
end;
WriteOut;
until ((CResult = Z_STREAM_END) and (strm.avail_out > 0));
--- End code ---
Navigation
[0] Message Index
[#] Next page