Recent

Author Topic: BlowFish busted in Laz 1.9 / Fpc 3.1.1  (Read 15898 times)

RedOctober

  • Sr. Member
  • ****
  • Posts: 475
BlowFish busted in Laz 1.9 / Fpc 3.1.1
« on: December 31, 2017, 07:59:44 pm »
My other post on this was implicating a Base64 problem.  I have done some testing and have discovered that the problem is actually BlowFish in Lazarus 1.9 / Fpc 3.1.1.

Method:

I installed both Laz 1.8 / Fpc 304 and Laz 1.9 / Fpc 3.1.1 on my development machine using fpcupdeluxe-x86_64-win64.exe v1.6.0m from https://github.com/newpascal/fpcupdeluxe/releases/tag/v1.6.0m


Reason:

I had to do this for testing purposes.  I had to upgrade my Fpc because FBAdmin component is busted in Fpc 304.  However, I'm still stuck, because BlowFish is now busted in Fpc 3.1.1.  I need a version combo where FBAdmin and BlowFish and Base64 work correctly.

The code below shows the results of my running the very same project in the two different environments.  My OS is Windows Server 2016.

Laz18, Fpc304 works perfectly.  No Memory Access errors.
Laz19, Fpc311 fails at the places indicated.
The comment "Correct" only means that it works, not that I am a Base64 or BlowFish expert.

Laz18 Fpc304: Input:  MyPlainText
Laz19 Fpc311: Input:  MyPlainText
               
First Part ----------
Encrypt using Blowfish, then Encode using Base64

Laz18 Fpc304: Bytes BlowFish Enc: b?#??Jo?d|??QK      // Correct
Laz19 Fpc311: Bytes BlowFish Enc: bµ#ëì¯Joàd|’'#14'ÙQK  // BlowFish goes wrong, right from the start

Laz18 Fpc304: Base64 Encoded: YrUj6+yvSm/gZHySDtlRSw==   // Correct
Laz19 Fpc311: Base64 Encoded: YsK1I8Orw6zCr0pvw6BkfOKAmQ7DmVFL  // Wrong, but understandable since the input was different

www.base64encode.org Encoded 304's output: Yj8jPz9Kbz9kfD8OP1FL                      // Disagrees with 304, but who cares.. 304 works
www.base64encode.org Encoded 311's output: YsK1I8Orw6zCr0pvw6BkfOKAmScjMTQnw5lRSw==  // Disagrees with 311, but input was different, so understandable


// 304 has correct output, so I can use it for other things.  It is both encrypted and MIME'd



// So now comes the time I want to reverse the process and get the original text MyPlainText...


// Second Part -----------
// Decode using Base64, then Decrypt using BlowFish

Laz18 Fpc304: Base64 Decoded: b?#??Jo?d|??QK              // Correct
Laz19 Fpc311: Base64 Decoded: bµ#ëì¯Joà d|’'#14'ÙQK  // Wrong, but understandable since the input was different

www.base64encode.org Decoded of 304's output: b#Jod|QK          // Disagrees with 304, but who cares.. 304 works
www.base64encode.org Decoded of 311's output: bµ#ëì¯Joàd|’ÙQK   // Disagrees with 311, but input was different, so understandable


Laz18 Fpc304: Base64 Dec: MyPlainText  // Correct
Laz19 Fpc311: Base64 Dec: Memory Access Error // Memory access error means I cannot use this in a production system.

 
// 304 works without error.  311 has errors, shown in comments above.

Thanks in advance for any help you can provide.  Code shown below.

Code: Pascal  [Select][+][-]
  1. procedure TfrmMain.Button2Click(Sender: TObject);
  2. var
  3.   mime_str_in, mime_str_out: String;
  4.   plain_text: String;
  5.  
  6.   enc_str, dec_str: RawByteString;
  7.   bfsh_enc_strm: TBlowFishEncryptStream;
  8.   bfsh_dec_strm: TBlowFishDecryptStream;
  9.   strm: TStringStream;
  10.  
  11. begin
  12.   master_pwd := '12345678';
  13.   plain_text := 'MyPlainText';
  14.   ShowMessage('plain_text: ' + plain_text);
  15.  
  16.   //--- Encrypt first
  17.   strm := TStringStream.Create('');
  18.   bfsh_enc_strm := TBlowfishEncryptStream.Create(master_pwd, strm);
  19.   bfsh_enc_strm.WriteAnsiString(plain_text);
  20.   bfsh_enc_strm.Flush;
  21.   enc_str := strm.DataString;
  22.  
  23.   ShowMessage('Encrypted String: ' + enc_str);
  24.  
  25.   // Now MIME encode it
  26.   mime_str_in := EncodeStringBase64(enc_str);
  27.   ShowMessage('mime_text: ' + mime_str_in);
  28.  
  29.   bfsh_enc_strm.Free;
  30.   strm.Free;
  31.   // Encryption/Encoding completed
  32.  
  33.   // Decode MIME first, before we decrypt
  34.   dec_str := DecodeStringBase64(PChar(mime_str_in.ToCharArray), False);
  35.  
  36.   ShowMessage('dec_str: ' + dec_str);
  37.  
  38.   if SameText(enc_str, dec_str) then
  39.     ShowMessage('enc_str and dec_str match')
  40.   else
  41.     ShowMessage('enc_str and dec_str DO NOT match');
  42.  
  43.   // Now, Decrypt
  44.   strm := TStringStream.Create(dec_str);
  45.   bfsh_dec_strm := TBlowfishDecryptStream.Create(master_pwd, strm);
  46.   dec_str := bfsh_dec_strm.ReadAnsiString;
  47.  
  48.   ShowMessage('dec_str: ' + dec_str);
  49.  
  50.   if SameText(plain_text, dec_str) then
  51.     ShowMessage('enc_str and dec_str match')
  52.   else
  53.     ShowMessage('enc_str and dec_str DO NOT match');
  54.  
  55.  
  56. end;
  57.  
  58.  



jamie

  • Hero Member
  • *****
  • Posts: 7702
Re: BlowFish busted in Laz 1.9 / Fpc 3.1.1
« Reply #1 on: December 31, 2017, 08:07:43 pm »
Take the unit fsource code to Blowfish that works, place it in your project folder using the
3.11 environment.

 Add this file to you project.. .

 Now it will use the version from 3.0.4//

 If it breaks at this point, then we know its something in 3.1.1 that is doing it.
The only true wisdom is knowing you know nothing

RedOctober

  • Sr. Member
  • ****
  • Posts: 475
Re: BlowFish busted in Laz 1.9 / Fpc 3.1.1
« Reply #2 on: December 31, 2017, 09:18:12 pm »
I am using Lazarus 1.9 / Fpc 3.1.1 with the following changes...

Steps:

1) I copied blowfish.pp from the 304 source code folder "C:\ProgramFilesCP\Lazarus_18_fpc_304\fpcsrc\packages\fcl-base\src"
and placed it in my project source code folder. I changed its name to blowfish304.pp and I added it to my project source. I did a Project Build (Shift+F9) and ran the project.  I had the same error as before.
2) I copied base64.pp  from the 304 source code folder and placed it in my project source code folder. I changed its name to base64304 and I added it to my project source. I did a Project Build (Shift+F9) and ran the project.  I had the same error as before.

The Error Text is:

~~~~~~~~~~~~~~~~~~~~~~~~~

[Debugger Exception Notification]

Project MyProject raised exception class 'EReadError' with message:
Stream read error

 At address 1000430D1


[Ignore this exception type]

[Break] [Continue]

~~~~~~~~~~~~~~~~~~~~~~~~~

Ok, now I am at a loss.

Is there a step that I am missing?  Is the 311 still using a compiled file that doesn't get changed by doing my steps above? (Like a .dcu file in Delphi?)
When I do a trace, I am stepped into the x304 copies of the source code files that are in my project.

Is there something in Lazarus 311 that is messing with the two functions above, and actually the functions are ok, but Lazarus is not?

Do you think my OS, Windows Server 2016 is messing with the functions?

I can see in my project, that the files were "compiled"... to their respective .o and .ppu files.

The only ogther thing I can think of is to start pulling the Classes, and SysUtils and one by one, all other units, over to my project source code folder and see at which point the error goes away.

molly

  • Hero Member
  • *****
  • Posts: 2330
Re: BlowFish busted in Laz 1.9 / Fpc 3.1.1
« Reply #3 on: December 31, 2017, 09:47:38 pm »
Is there something in Lazarus 311 that is messing with the two functions above, and actually the functions are ok, but Lazarus is not?
That is what i've been trying to tell you when i suggested you to do some tests (in the other thread).

_always_ reduce the number of equations (= get rid of lazarus). If 3.1.1 fpc compiler is able to reproduce the issue then it is fpc at fault, if not then it might be lazarus at fault.

As i mentioned before (or at least concluded/suggested by user jamie) there seems to be a utf8 issue somewhere. Don't use unicode editors, or exotic code-pages, don't use whatever exotic string-types and try to compile such project with fpc commandline compiler to see if that solves the problem.

I would be more than glad to build myself a trunk compiler and test it myself, but i'm currently in no position to be able to (ifor me that has to wait to next year  :) )

RedOctober

  • Sr. Member
  • ****
  • Posts: 475
Re: BlowFish busted in Laz 1.9 / Fpc 3.1.1
« Reply #4 on: December 31, 2017, 10:05:01 pm »
Thanks Molly.  It may be quicker just to wait until 3.1.2 comes out, as that was the target for the Stream fix you linked me to earlier. (I've never used command line FPC on it's own.  I've been using Delphi since 1996)

I'll proceed with the parts of my project that don't use encryption, but I won't be able to release it without this functionality.

molly

  • Hero Member
  • *****
  • Posts: 2330
Re: BlowFish busted in Laz 1.9 / Fpc 3.1.1
« Reply #5 on: December 31, 2017, 10:17:48 pm »
Your efforts so far are appreciated RedOctober . If nobody reports it then the error (if there is actually an error somewhere) will stay. Also in fpc 3.0.6/3.2.0 release.

I'll try to see if i'm able to reproduce with trunk compiler when i'm able to. But if i would fail in doing so then i won't submit a bug-report (on your behalf). I have no idea how to reproduce your issue atm (my current lazarus and fpc trunk-builds are too old already to make a fair comparison).

Please continue developing your application (working around the issue) and have a happy new year !  :)
« Last Edit: December 31, 2017, 10:21:19 pm by molly »

RedOctober

  • Sr. Member
  • ****
  • Posts: 475
Re: BlowFish busted in Laz 1.9 / Fpc 3.1.1
« Reply #6 on: December 31, 2017, 10:28:17 pm »
I will!  Happy New Year Molly!

For what it's worth... ALL dev environments have similar issues with versions and bugs. I'm very much enjoying working with Lazarus.  It's just so simple, quick, and 99% of the defaults for the IDE are what I would have picked anyway, so very little "setup" needs to be done after each install.

I also like the way fpcupdeluxe can install an "isolated" non-intrusive copy (copies) of what ever version combos you want.  You guys at Lazarus and FPC have a good thing going here and I want to be a part of it.


taazz

  • Hero Member
  • *****
  • Posts: 5368
Re: BlowFish busted in Laz 1.9 / Fpc 3.1.1
« Reply #7 on: December 31, 2017, 10:30:29 pm »
I have no idea where the problem might but I have two comments to add to the conversation.
1) There is no need to copy the rest of the FPC rtl just to verify an exception, a far better approach is to copy the working code to a new unit with a unique name.
2) When you have that kind of problems it is far better to provide us with a simple minimal example for us to compile and run to see the exception for our self's. Make sure you provide the required data as well and expected results eg. a encrypted file or string the password to decrypt it and the expected unencrypted string/file. We will take a close look for you.
sorry but as things are there is no way to know where is the bug and what to report.
Good judgement is the result of experience … Experience is the result of bad judgement.

OS : Windows 7 64 bit
Laz: Lazarus 1.4.4 FPC 2.6.4 i386-win32-win32/win64

molly

  • Hero Member
  • *****
  • Posts: 2330
Re: BlowFish busted in Laz 1.9 / Fpc 3.1.1
« Reply #8 on: December 31, 2017, 10:37:47 pm »
Hi taazz,

User RedOctober forgot to provide a complete example here but can be found in this thread.

in short: TS has to use trunk because of other issues that where fixed there, but trunk seems to introduce this 'bug' with regards to blowfish for him. I can't reproduce with 3.0.4 compiler or my (outdated) 3.1.1 compiler.
« Last Edit: December 31, 2017, 10:41:24 pm by molly »

RedOctober

  • Sr. Member
  • ****
  • Posts: 475
Re: BlowFish busted in Laz 1.9 / Fpc 3.1.1
« Reply #9 on: December 31, 2017, 10:49:49 pm »
Hi Tazz and Molly,

I included the source code to reproduce the problem in the top post of this thread. It's under the text:  "Code shown below."

Is the code tag(s) not rendering properly? 

From your comments, it occurred to me that maybe you are not being shown the included source code section.

Code: Pascal  [Select][+][-]
  1. Like This


taazz

  • Hero Member
  • *****
  • Posts: 5368
Re: BlowFish busted in Laz 1.9 / Fpc 3.1.1
« Reply #10 on: December 31, 2017, 10:53:22 pm »
Hi taazz,

User RedOctober forgot to provide a complete example here but can be found in this thread.

in short: TS has to use trunk because of other issues that where fixed there, but trunk seems to introduce this 'bug' with regards to blowfish for him. I can't reproduce with 3.0.4 compiler or my (outdated) 3.1.1 compiler.
Oh Ok. I got the code from the the first post in the linked forum thread and I'll take a closer look tomorrow morning (or later tonight after new year's celebrations) but after a quick look on the code I see one problematic line
Code: Pascal  [Select][+][-]
  1. function EncryptItB64(const pwd, plain_text: String): String;
  2. var
  3.   bfsh: TBlowFishEncryptStream;
  4.   strm: TStringStream;
  5.   str_b64, byt_str: String;
  6. begin
  7.   Result := '';
  8.   str_b64 := '';
  9.  
  10.   if Length(plain_text) = 0 then
  11.     Exit;
  12.  
  13.   strm := nil;
  14.   bfsh := nil;
  15.   try
  16.     strm := TStringStream.Create('');
  17.     bfsh := TBlowfishEncryptStream.Create(pwd, strm);
  18.     bfsh.WriteAnsiString(plain_text);
  19.  
the method writeansistring adds the string length in front of the string which it self is encrypted with blowfish and of course readansistring expects the stream position/cursor to be on the length of the string that follows but since it is encrypted the result will always be wrong. use the write/read method instead while encrypting and only use the writeansistring/readansistring after the encryption.

The rest after I tested the provided code.

PS:

 HAPPY NEW YEAR EVERYONE.
« Last Edit: December 31, 2017, 10:56:29 pm by taazz »
Good judgement is the result of experience … Experience is the result of bad judgement.

OS : Windows 7 64 bit
Laz: Lazarus 1.4.4 FPC 2.6.4 i386-win32-win32/win64

molly

  • Hero Member
  • *****
  • Posts: 2330
Re: BlowFish busted in Laz 1.9 / Fpc 3.1.1
« Reply #11 on: December 31, 2017, 10:56:05 pm »
RedOctober:
uhm... now i can see it. no idea why i missed it.

Just to make sure. It is common to make a small test example (like you did) and then "publish" your project. All necessary files are then put into a special directory by lazarus and you can zip that up and attach to a forum post.

That way someone else if able to download the zip and load your project. Usually that is enough for someone else to create exact similar conditions as yours. For example, if you saved your source in unicode then this is missed when you copy-paste your code into a forum message.

jamie

  • Hero Member
  • *****
  • Posts: 7702
Re: BlowFish busted in Laz 1.9 / Fpc 3.1.1
« Reply #12 on: December 31, 2017, 11:07:16 pm »
Is it possible for you to install the 32 bit version of Lazarus and try it there?

 its very possible there is an oversight use of a Pointers casting etc...
The only true wisdom is knowing you know nothing

molly

  • Hero Member
  • *****
  • Posts: 2330
Re: BlowFish busted in Laz 1.9 / Fpc 3.1.1
« Reply #13 on: December 31, 2017, 11:09:45 pm »
HAPPY NEW YEAR EVERYONE.
Happy new year !!! taaz, and everyone else !!  :)

RedOctober

  • Sr. Member
  • ****
  • Posts: 475
Re: BlowFish busted in Laz 1.9 / Fpc 3.1.1
« Reply #14 on: December 31, 2017, 11:12:02 pm »
Ooohhhh... so THAT's what "Publish" means.  Ha! Now I know.  Good explanation Molly. 

About the . xAnsiString  good point.  I tried many different types of strings to break 304 (which it never did) and to fix 311 (which it never did).   Also, there are some functions that exist on the Blowfish stream that are actually not allowed to be used.  I think .Read was one of them.  I used .WriteAnsiString and .ReadAnsiString thinking they would work as a "pair".(what ever one added, the other would take away)... also this exact code works in 304, so I left it.  :)

 

TinyPortal © 2005-2018