TBase64 Encoding-Decoding

[Relative]

Hi Coders!

I would like to make two DLL function:
1.) EnCode a PChar text with TBase64EncodingStream
2.) DeCode a PChar text with TBase64DecodingStream

Here it is my code:

Code: [Select]

library code_v004;

uses
  sysutils, Classes, base64;

var
   DecodedStream: TStringStream;
   EncodedStream: TStringStream;
   Encoder: TBase64EncodingStream;
   Decoder: TBase64DecodingStream;

// enCode
function enCode(x: PChar): PChar; stdcall;
begin
     DecodedStream := TStringStream.Create(x);
     EncodedStream := TStringStream.Create('');
     Encoder       := TBase64EncodingStream.Create(EncodedStream);
     Encoder.CopyFrom(DecodedStream, DecodedStream.Size);
     result := PChar(EncodedStream.DataString);
end;

// deCode
function deCode(x: PChar): PChar; stdcall;
begin
     EncodedStream := TStringStream.Create(x);
     DecodedStream := TStringStream.Create('');
     Decoder       := TBase64DecodingStream.Create(EncodedStream);
     DecodedStream.CopyFrom(Decoder, Decoder.Size);
     result := PChar(DecodedStream.DataString);
end;

exports
  enCode, deCode;

begin
end.
First look, it seems working.

But with every text!

Try this string: Háromszéki Zsolt
The encoded string will be: SOFyb21zeulraSBac29s

But really, the encoded string isn't the whole string. Four characters (dA0K) miss from the end. The real encoded string should be: SOFyb21zeulraSBac29sdA0K

With other strings, the enCode() function works properly. But  what about the 'Háromszéki Zsolt' text? Why doesn't work with it?

Do you have any idea?

Thank you in advance.

Relative

[marcov]

Hi Coders!

But really, the encoded string isn't the whole string. Four characters (dA0K) miss from the end. The real encoded string should be: SOFyb21zeulraSBac29sdA0K

Stream classes and classes that process them might cache data. Therefore you must either flush or destruct streams, to force all remaining data to be flushed out. This can be a little tricky. Try these:
 

Code: [Select]

function DecodeStringBase64(const s:string):String;

var instream,outstream : TStringStream;
    decoder : TBase64DecodingStream;
begin
  instream:=TStringStream.Create(s);
  try
    outstream:=TStringStream.Create('');
    try
      decoder:=TBase64DecodingStream.create(instream,bdmmime);
      try
         outstream.copyfrom(decoder,decoder.size);
         outstream.position:=0;
         result:=outstream.readstring(outstream.size);
      finally
        decoder.free;
        end;
    finally
     outstream.free;
     end;
  finally
    instream.free;
    end;
end;


function EncodeStringBase64(const s:string):String;

var outstream : TStringStream;
    encoder : TBase64EncodingStream;
begin
  outstream:=TStringStream.Create('');
  try
    encoder:=TBase64EncodingStream.create(outstream);
    try
      encoder.write(s[1],length(s));
    finally
      encoder.free;
      end;
    outstream.position:=0;
    result:=outstream.readstring(outstream.size);
  finally
    outstream.free;
    end;
end;

Starting with the upcoming 2.4.4, the above two procedures will be standard in unit base64 though, so you won't need to handcode them anymore

[Relative]

Thank you Marcov.

Is it possible to use PChar without String? I replaced the 'string' words in the code, but I got this error message:

'Error: Incompatible types: got "AntiString" expected PChar'

[marcov]

Thank you Marcov.

Is it possible to use PChar without String? I replaced the 'string' words in the code, but I got this error message:

'Error: Incompatible types: got "AntiString" expected PChar'

It might be possible, but it doesn't make sense, since it will be converted one step later anyway.

[Relative]

All right!

It's working!

Thank you Marcov!

[circular]

I wonder where the string gets freed. I suppose this never happens. You could store the result in a global variable, so that there would be only one string in memory. But if multiple threads ask for a string, this could give mixed results. Maybe with a GlobalAlloc or something like that.

Anyway if it works, that's good for you, but I suppose there is a memory leak.

[eny]

You could store the result in a global variable...

This is exactly where it went wrong in the first place: using globals that are not properly freed hence the invalid string result.
In marcov's example all variable handling is done locally (as it should be), all objects are freed (the data flushed) and the results therefore are correct.

[circular]

I don't understand. If the result is freed, then the PChar points to an freed memory. This is rather strange, no?

There is another solution : the caller allocate a buffer where the result is stored.

[Relative]

This code works:

Code: [Select]

library code_v006;

uses
  Classes, base64;

// deCode
function dC(s:PChar): PChar; stdcall;
var instream,outstream : TStringStream;
    decoder : TBase64DecodingStream;
begin
  instream:=TStringStream.Create(s);
  try
    outstream:=TStringStream.Create('');
    try
      decoder:=TBase64DecodingStream.create(instream,bdmmime);
      try
         outstream.copyfrom(decoder,decoder.size);
         outstream.position:=0;
         result:=PChar(outstream.readstring(outstream.size));
      finally
        decoder.free;
        end;
    finally
     outstream.free;
     end;
  finally
    instream.free;
    end;
end;

// enCode
function eC(s:PChar): PChar; stdcall;
var outstream : TStringStream;
    encoder : TBase64EncodingStream;
begin
  outstream:=TStringStream.Create('');
  try
    encoder:=TBase64EncodingStream.create(outstream);
    try
      encoder.write(s[1],length(s));
    finally
      encoder.free;
      end;
    outstream.position:=0;
    result:=PChar(outstream.readstring(outstream.size));
  finally
    outstream.free;
    end;
end;

exports
  eC, dC;

begin
end.

[eny]

This code works:

I'm fairly sure it doesn't.
No missing first characters?

Less is more:

Code: Pascal  [Select][+][-]

1.   function eC(s:PChar): PChar; stdcall;
2.   begin
3.     with TBase64EncodingStream.create(TStringStream.Create('')) do
4.     try
5.       SourceOwner := true;
6.       write(s^,length(s));
7.       result := PChar(TStringStream(Source).DataString)
8.     finally
9.       free;
10.     end;
11.   end;

[Relative]

I'm fairly sure it doesn't.
No missing first characters?

Oh, yes, it does! LOL I was just looking for the end of the text...

Less is more

It seems much more better! Thank you Eny!

[eny]

Slightly off topic: IMHO from an architectural point of view it would be better if the LCL would have some sort of 'TStreamFilter' class that accepts an input and output stream and an abstract 'TFilter' class. It copies all data from input to output stream while passing it through the TFilter object.
Descendants of TFilter (for example a 'TBase64Encoder' or 'TBase64Decoder') would implement the required 'filtering'.
That would make it very easy to implement new filters (other encoders, decoders, parsers, whatever).

[marcov]

Slightly off topic: IMHO from an architectural point of view it would be better if the LCL would have some sort of 'TStreamFilter' class that accepts an input and output stream and an abstract 'TFilter' class. It copies all data from input to output stream while passing it through the TFilter object.
Descendants of TFilter (for example a 'TBase64Encoder' or 'TBase64Decoder') would implement the required 'filtering'.
That would make it very easy to implement new filters (other encoders, decoders, parsers, whatever).

I don't understand why that code needs a different class. Streams are already abstracted.

[eny]

I don't understand why that code needs a different class. Streams are already abstracted.

You're right, streams are OK.
The idea is to abstract the encoding from streams, something like below.

Code: Pascal  [Select][+][-]

1. type
2.   // Abstract base class for filters (encoders etc.)
3.   TFilter = class
4.   public
5.     // 2 be overriden by concrete classes
6.     procedure process(var pIn; var pOut; const pSize: integer); virtual; abstract;
7.   end;
8.  
9.   // Concrete class to read all data from instream, process through filter and write
10.   // to an outstream.
11.   TStreamFilter = class
12.   private
13.     FFilter: TFilter;
14.   public
15.     constructor Create(const pFilter: TFilter); // Register filter to use
16.     procedure Copy(pIn, pOut: TStream);         // Copy instream to outstream via filter
17.   end;
18.  
19.   TBase64Encoder = class(TFilter)
20.   public
21.     // Override 'process' to do the encoding
22.     procedure process(var InBuf; var OutBuf; const pSize: integer); override;
23.   end;
24.  
25.   TBase64Decoder = class(TFilter)
26.   public
27.     // Override 'process' to do the decoding
28.     procedure process(var InBuf; var OutBuf; const pSize: integer); override;
29.   end;
30.  
31.   // ... whatever new filters may come...
32.  
33.   var
34.     ins, outs: TStringStream;
35.     sf       : TStreamFilter;
36.  
37.   //...
38.   ins := TStringStream.Create('<2 be encoded text>');
39.   outs := TStringStream.Create('');
40.   sf := TStreamFilter.Create(TBase64Encoder.Create);
41.   sf.Copy(ins,outs);
42.  
43.   // ...
44.   ins := TStringStream.Create('<encoded text>');
45.   outs := TStringStream.Create('');
46.   sf := TStreamFilter.Create(TBase64Decoder.Create);
47.   sf.Copy(ins,outs);
48.  

[marcov]

I don't understand why that code needs a different class. Streams are already abstracted.

You're right, streams are OK.
The idea is to abstract the encoding from streams, something like below.

I don't really see what this division gains. Sure, you can allocate an extra class around what is basically a single procedure, but what is the point?

Moreover there are other problems, streams are divided between seekable and non seekable, handle based and non handle based streams. In many operations you might want to use these properties, if available.