With is the best way to compress string ?

[seba22]

Welcome,

I have very long text string. (String contain many repeated sentences.)
I'm wondering is it possible to compress it using  IdCompressorZLib1 and display to user in base64 encoded form ?

And reverse way,  user insert base64 encoded, base64 decode then uncompress to get native form ?


Is there any way (win / linux) ?

I saw this one http://wiki.lazarus.freepascal.org/bzip2lib  but it require aditional libraries, and also example show how to create save file.



In php i have something like this:

Code: [Select]

<?php
$compressed = base64_encode(gzcompress('Compress me', 9));
echo $compressed;
?>
Regards


 

[Leledumbo]

Maybe a combination of paszlib and base64 unit?

[Laksen]

Here's an example based on paszlib and the base64 unit

Code: [Select]

uses zstream, base64;
...
function GzBase64(const s: string): string;
var OutStream, InpStream, GzStream, b64Stream: TStream;
begin
   OutStream := TStringStream.Create('');
   try
      b64Stream := TBase64EncodingStream.Create(OutStream);
      try
         GzStream := Tcompressionstream.create(clmax,b64Stream);
         try
            InpStream := TStringStream.Create(s);
            try
               // Copy input stream
               GzStream.CopyFrom(InpStream,InpStream.Size);

               // Flush compression and then base64
               Tcompressionstream(GzStream).flush;
               TBase64EncodingStream(b64Stream).flush;

               // Read out datastring :)
               result := TStringStream(OutStream).DataString;
            finally
               InpStream.Free;
            end;
         finally
            GzStream.Free;
         end;
      finally
         b64Stream.Free;
      end;
   finally
      OutStream.Free;
   end;
end;


[seba22]

Welcome,
Thank You for code.


I got error at

Code: [Select]

TBase64EncodingStream(b64Stream).flush;It says that TBase64EncodingStream don't have flush member.


And the most important thing is how to uncompress ?

Code: [Select]

function GzBase64(const s: string): string;
var OutStream, InpStream, GzStream, b64Stream: TStream;
begin
   OutStream := TStringStream.Create('');
   try
      b64Stream := TBase64EncodingStream.Create(OutStream);
      try
         GzStream := Tcompressionstream.create(clmax,b64Stream);
         try
            InpStream := TStringStream.Create(s);
            try
               // Copy input stream
               GzStream.CopyFrom(InpStream,InpStream.Size);

               // Flush compression and then base64
               Tcompressionstream(GzStream).flush;
            //   TBase64EncodingStream(b64Stream).flush;

               // Read out datastring :)
               result := TStringStream(OutStream).DataString;
            finally
               InpStream.Free;
            end;
         finally
            GzStream.Free;
         end;
      finally
         b64Stream.Free;
      end;
   finally
      OutStream.Free;
   end;
end;
Regards

I'm trying to rewrite it.

[/b]

Code: [Select]

function GzBase64(const s: string): string;
var OutStream,CompressStream: TStream;
begin
OutStream := TStringStream.Create('');

CompressStream := TBzip2CompressStream.Create(OutStream);
CompressStream.Write(Pointer(S)^, Length(S));

result := form1.IDEncoderMIME1.Encode (TStringStream(OutStream).DataString);

end; 
Why this don't work ?

Im trying

Code: [Select]

temp:=GzBase64('testtesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttest');
   showmessage(temp);   
Result is empty string ?

[Laksen]

Flushing is a rather important part. Most compression streams take as much data as they can and only write when you flush or destroy the stream. That's the case for TBzip2CompressStream, TBase64EncodingStream and Tcompressionstream

I think some old version of FPC didn't have flush of TBase64EncodingStream visible outside private, but that's changed now. Either way it usually flushes on Destroy.

Which means it should be like this:

Code: [Select]

function GzBase64(const s: string): string;
var OutStream, InpStream, GzStream, b64Stream: TStream;
begin
   OutStream := TStringStream.Create('');
   try
      b64Stream := TBase64EncodingStream.Create(OutStream);
      try
         GzStream := Tcompressionstream.create(clmax,b64Stream);
         try
            InpStream := TStringStream.Create(s);
            try
               // Copy input stream
               GzStream.CopyFrom(InpStream,InpStream.Size);
            finally
               InpStream.Free;
            end;
         finally
            GzStream.Free;
         end;
      finally
         b64Stream.Free;
      end;
      result := TStringStream(OutStream).DataString;
   finally
      OutStream.Free;
   end;
end;

And the code you posted should be something like this:

Code: [Select]

function GzBase64(const s: string): string;
var OutStream,CompressStream: TStream;
begin
   OutStream := TStringStream.Create('');
   try
      CompressStream := TBzip2CompressStream.Create(OutStream);
      try
         CompressStream.WriteBuffer(S[1], Length(S));
      finally
         CompressStream.Free;
      end;
      result := form1.IDEncoderMIME1.Encode(TStringStream(OutStream).DataString);
   finally
      OutStream.Free;
   end;
end;


[seba22]

Thank You, working perfect.


Today i wrote decompression, i don't know if i done it in good way, but it working.
 

Code: [Select]

function unGzBase64(const s: string): string;
var OutStream,deCompressStream: TStream;
     SL:TStringList;

begin
if(s='')
then
begin
Result:='';
abort;
end;

 SL := TStringList.Create;
 OutStream := TStringStream.Create(form1.IdDecoderMIME1.DecodeString(s));
 DecompressStream := TBzip2DecompressStream.Create(OutStream);

 try
  SL.LoadFromStream(DecompressStream);
  Result:=SL.Text;
  finally
    DecompressStream.Free;
    OutStream.Free;
    SL.Free;
  end;


end;

(i make simple test)

Code: [Select]

temp:=GzBase64('testtesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttest');
  showmessage(unGzBase64(temp));
 
And working well

[seba22]

Update

I move that code to windows, and compile.

Compile ok, but when i want compress string.

It throw error at

Code: [Select]

CompressStream.WriteBuffer(S[1], Length(S));
         

Code: [Select]

Error is: External SIGSEGV
On linux working well.

I copy bzip2.dll  into project directory.


I don't know what can cause problem.

I compile on 2 different windows pc (2k3 server and windows 7) both not working :/

[seba22]

UPDATE

Code: [Select]

function GzBase64(const s: string): string;
var OutStream, InpStream, GzStream, b64Stream: TStream;
begin
   OutStream := TStringStream.Create('');
   try
      b64Stream := TBase64EncodingStream.Create(OutStream);
      try
         GzStream := Tcompressionstream.create(clmax,b64Stream);
         try
            InpStream := TStringStream.Create(s);
            try
               // Copy input stream
               GzStream.CopyFrom(InpStream,InpStream.Size);
            finally
               InpStream.Free;
            end;
         finally
            GzStream.Free;
         end;
      finally
         b64Stream.Free;
      end;
      result := TStringStream(OutStream).DataString;
   finally
      OutStream.Free;
   end;
end;
function unGzBase64(const s: string): string;
var OutStream,deCompressStream: TStream;
     SL:TStringList;

begin
if(s='')
then
begin
Result:='';
abort;
end;

 SL := TStringList.Create;
 OutStream := TStringStream.Create(form1.IdDecoderMIME1.DecodeString(s));
 DecompressStream := TDecompressionStream.Create(OutStream);

 try
  SL.LoadFromStream(DecompressStream);
  Result:=SL.Text;
  finally
    DecompressStream.Free;
    OutStream.Free;
    SL.Free;
  end;


end;
This working on linux and windows, tested