SHA-1 hash on i386 Linux

[MarkMLl]

If anybody has FPC trunk on i386 Linux, please could they check the sha1test program from the hash package.

On x86_64 I get

Code: Text  [Select][+][-]

1. /usr/local/src.fpc/3.2.2/packages/hash/examples$ ./sha1test
2. Performing basic SHA-1 test...
3. Basic SHA-1 tests passed
4.  

while on i386 I get

Code: Text  [Select][+][-]

1. /usr/local/src.fpc/3.2.2/packages/hash/examples$ ./sha1test
2. Performing basic SHA-1 test...
3. SHA-1 tests failed: 7
4.  

and I can confirm that a program trying to use the package gets results at variance with Linux's sha1sum.

I'll try to get something later than 3.2.2 running here later in the day, but it won't be for at least some hours and if this is a genuine bug getting it reported promptly is probably in order.

Updated: appears to be the same in 3.2.0, but basic test passes in FPC 3.0.4:

Code: Text  [Select][+][-]

1. /usr/local/src.fpc/3.0.4/packages/hash/examples$ ./sha1test
2. Performing basic SHA-1 test...
3. Basic SHA-1 tests passed
4.  

MarkMLl

[Thaddy]

I do not even see a file sha1test.pp or pas or lpr in the examples directory, I do see one in tests but with a different name.
Cross-compiled from x86_64-win64 to i386-win32 all tests pass in main, as they do for 64 bit.
Is your file a remnant of a previous install? That should be shown by git or svn. The file you mention is NOT in the fpcsrc directories.
Without that file I can't test linux 64/32

The examples are all very short so you can safely repost that ghost example here.

If it is your own code, well, check ... The officiial tests and examples all pass.

[MarkMLl]

It's from fpcbuild-3.2.2.tar.gz.

Code: Text  [Select][+][-]

1. /usr/local/src.fpc$ find . -name 'sha1test*'
2. ./fpcbuild/fpcbuild/fpcsrc/tests/test/packages/hash/sha1test.pp
3. ./fpcbuild/fpcbuild/fpcsrc/packages/hash/examples/sha1test.pp
4. ./fixes-3.2.3/fpc/tests/test/packages/hash/sha1test.pp
5. ./fixes-3.2.3/fpc/packages/hash/examples/sha1test.pp
6. ./fpc-3.2.0rc1/tests/test/packages/hash/sha1test.pp
7. ./fpc-3.2.0rc1/packages/hash/examples/sha1test.pp
8. ./fpcbuild-3.0.4/fpcsrc/tests/test/packages/hash/sha1test.pp
9. ./fpcbuild-3.0.4/fpcsrc/packages/hash/examples/sha1test.pp
10. ./fpcbuild-3.0.2/fpcsrc/tests/test/packages/hash/sha1test.pp
11. ./fpcbuild-3.0.2/fpcsrc/packages/hash/examples/sha1test.pp
12. ./fpc-trunk/fpc/tests/test/packages/hash/sha1test.pp
13. ./fpc-trunk~/fpc/tests/test/packages/hash/sha1test.pp
14. ./fpcbuild-3.2.2/fpcsrc/tests/test/packages/hash/sha1test.pp
15. ./fpcbuild-3.2.2/fpcsrc/packages/hash/examples/sha1test.pp
16. ./fpcbuild-3.2.2/fpcsrc/packages/hash/examples/sha1test
17. ./fpcbuild-3.2.2/fpcsrc/packages/hash/examples/units/x86_64-linux/sha1test.o
18. ./fpcbuild-3.0.0/fpcsrc/tests/test/packages/hash/sha1test.pp
19. ./fpcbuild-3.0.0/fpcsrc/packages/hash/examples/sha1test.pp
20. ./fpcbuild-2.6.4/fpcsrc/tests/test/packages/hash/sha1test.pp
21. ./fpcbuild-2.6.4/fpcsrc/packages/hash/examples/sha1test.pp
22. ./fpcbuild-3.2.0/fpcsrc/tests/test/packages/hash/sha1test.pp
23. ./fpcbuild-3.2.0/fpcsrc/packages/hash/examples/sha1test.pp
24.  

 We've been though this before Thaddy, when you accused me of using improper sources or an unreleased OS.

MarkMLl

[Thaddy]

I am not accusing you of anything, It is just that I check out the sources - and build for a lot of platforms - almost daily, certainly weekly and that file is not there.... it is simply a full git repository pull request. And trying to test for you... Strange remark. FIA will not agree with my real thoughts.
 
[edit]
I also tested a git clone: file is not there, can't test.

Maybe it ended up there by the packager of the tar? In the source tree it is not. And should never happen, I thought that was automated.
Simply add the file here, as I wrote all these examples are short. Now I know J.S.

[MarkMLl]

Maybe it ended up there by the packager of the tar? In the source tree it is not. And should never happen, I thought that was automated.
Simply add the file here, as I wrote all these examples are short.

Below is from 3.2.2 However as I've said, application code written using SHA-1 from that package delivers the wrong result on i386: x86_64 corresponds to Linux's sha1sum but not i386.

MarkMLl

[Thaddy]

OK, tested:

Code: Bash  [Select][+][-]

1. C:\Users\thadd\Downloads>sha1test
2. Performing basic SHA-1 test...
3. Basic SHA-1 tests passed

Cross compiled for 32 bit windows i386

Code: Bash  [Select][+][-]

1. ppcross386
2. Free Pascal Compiler version 3.3.1-16840-gbb27442b19 [2024/11/08] for i386

[Fibonacci]

I found this file on my PC and its a little different

1. string -> rawbytestring
2. PChar -> PAnsiChar

Check if this one works.

Code: Pascal  [Select][+][-]

1. program sha1test;
2.  
3. {$mode objfpc}
4. {$h+}
5.  
6. uses
7.   {$ifdef unix}
8.   cwstring,
9.   {$endif}
10.   SysUtils, sha1;
11.  
12. function performTest: cardinal;
13.  
14. // Runs test and returns result code (0=success)
15. var
16.   s, sdig: rawbytestring;
17.   i: integer;
18.   ctx: TSHA1Context;
19.   d: TSHA1Digest;
20. begin
21.   result := 0;
22.   sdig := SHA1Print(SHA1String('abc'));
23.   if sdig <> 'a9993e364706816aba3e25717850c26c9cd0d89d' then
24.     result := result or 1;
25.  
26.   sdig := SHA1Print(SHA1String('abcdbcdecdefdefgefghfghighijhijkijkljklmklmnlmnomnopnopq'));
27.   if sdig <> '84983e441c3bd26ebaae4aa1f95129e5e54670f1' then
28.     result := result or 2;
29.  
30.   // SHA-1 of a million 'a' symbols
31.   SetLength(s, 1000);
32.   for i := 1 to 1000 do s[i] := 'a';
33.   SHA1Init(ctx);
34.   for i := 0 to 999 do
35.     SHA1Update(ctx, PAnsiChar(s)^, 1000);
36.   SHA1Final(ctx, d);
37.   sdig := SHA1Print(d);
38.   if sdig <> '34aa973cd4c4daa4f61eeb2bdbad27316534016f' then
39.     result := result or 4;
40. end;
41.  
42. var
43.   StartTime, EndTime: TDateTime;
44.   code: cardinal;
45. begin
46.   writeln('Performing basic SHA-1 test...');
47.   code:=performTest;
48.  
49.   if code = 0 then
50.     writeln('Basic SHA-1 tests passed')
51.   else
52.   begin
53.     writeln('SHA-1 tests failed: ', code);
54.   end;
55.   Halt(code);  
56. end.

[Thaddy]

It is not finding it on your PC this is about a file that is NOT in the current code repository.
And the file Mark provided simply works as expected.

[MarkMLl]

It is not finding it on your PC this is about a file that is NOT in the current code repository.
And the file Mark provided simply works as expected.

Thanks, in that case I'll assume it's not worth raising a bug report.

But I'm sure not using it in my code after finding that some combinations or target and version are broken.

MarkMLl

[Thaddy]

I just ran a suite of tests on that file and all others  and noone failed on i386-linux
The changes that Fibonacci mentioned may be the cause, but that also means there is something wrong at your side OR in the tar.
I did my best, now closing this.

[tetrastes]

I do not even see a file sha1test.pp or pas or lpr in the examples directory, I do see one in tests but with a different name.

Just in case: https://gitlab.com/freepascal.org/fpc/source/-/commit/4d126cbe5732453a94b67824d5cd137a3a4e9f1b.

Is your file a remnant of a previous install? That should be shown by git or svn. The file you mention is NOT in the fpcsrc directories.

F.e. https://gitlab.com/freepascal.org/fpc/source/-/tree/release_3_2_4-branch/packages/hash/examples?ref_type=heads.

[Thaddy]

I know that and I ran all tests. I already mentioned the rename.
In its current state the sourcetree for this particular problem does not contain any fails.

[MarkMLl]

I just ran a suite of tests on that file and all others  and noone failed on i386-linux
The changes that Fibonacci mentioned may be the cause, but that also means there is something wrong at your side OR in the tar.
I did my best, now closing this.

Well, what I see is that both that example and my own code were problematic on 3.2.x i386. And as a result there is absolutely no way that I'm using SHA-1 in code that I'll either put on Github or offer to the project, since doing so would both cause me extra support work and risk harming FPC's reputation.

And I make that decision in full knowledge that I risk your derision for continuing to use MD5, albeit in a noncritical context.

MarkMLl

[tetrastes]

I can confirm the issue for fpc 3.2.2 i386-linux. The problem is that sha1.pp includes sha1i386.inc with asm code:

Code: Pascal  [Select][+][-]

1. // Normally, if an optimized version is available for OS/CPU, that will be used
2. // Define to use existing unoptimized implementation
3. { the assembler implementation does not work on darwin }
4. {$ifdef darwin}
5. {$DEFINE SHA1PASCAL}
6. {$endif darwin}
7. . . .
8. {$IF (NOT(DEFINED(SHA1PASCAL))) and (DEFINED(CPU386)) }
9. // Use assembler version if we have a suitable CPU as well
10. // Define SHA1PASCAL to force use of original reference code
11. {$i sha1i386.inc}
12. {$ELSE}
13. // Use original version if asked for, or when we have no optimized assembler version

which doesn't work not only on darwin, but on linux also.
It is fixed in fixes_3_2 and release_3_2_4-branch: https://gitlab.com/freepascal.org/fpc/source/-/commit/96501336d76ea3f7c9b1fa8b7f55bad5c318df23, and in trunk in sha1i386.inc. 

[MarkMLl]

I can confirm the issue for fpc 3.2.2 i386-linux. The problem is that sha1.pp includes sha1i386.inc with asm code:

Thanks for that, so I'm somewhat less crazy (or at least cranky) than Thaddy :-)

Rationale at https://forum.lazarus.freepascal.org/index.php/topic,69103.0.html , i.e. the intention is to detect and deter unwise editing of a machine-generated file rather than implement something banking-grade, which suggests that MD5 is adequate. Having seen a problem with SHA-1, even if difficult to reproduce, strongly suggests that it's best not used.

So is this something that needs to be bug-reported, or do later versions fix it?

MarkMLl