Recent

Author Topic: Offical launch of the 1 Billion Row Challenge in Object Pascal  (Read 20995 times)

Gustavo 'Gus' Carreno

  • Hero Member
  • *****
  • Posts: 1132
  • Professional amateur ;-P
Re: Offical launch of the 1 Billion Row Challenge in Object Pascal
« Reply #30 on: March 22, 2024, 03:49:28 pm »
Hey Y'All,

After some extensive testing from our dear @paweld:
  • Tested with RAMDisk, SSD and HDD
  • Tested with the original input file containing only 400 weather stations

He came to the conclusion that there's really nothing to see between all of the above data containers.

I've eliminated the HDD column in the results.

Cheers,
Gus
Lazarus 3.99(main) FPC 3.3.1(main) Ubuntu 23.10 64b Dark Theme
Lazarus 3.0.0(stable) FPC 3.2.2(stable) Ubuntu 23.10 64b Dark Theme
http://github.com/gcarreno

Gustavo 'Gus' Carreno

  • Hero Member
  • *****
  • Posts: 1132
  • Professional amateur ;-P
Re: Offical launch of the 1 Billion Row Challenge in Object Pascal
« Reply #31 on: March 22, 2024, 04:23:12 pm »
Hey Y'All,

Because @paweld is the bloody GOAT* that we all know, we now have a switch on the generator to give us only 400 weather stations on the input file.

SHA256 for the 400 stations file:
Code: Text  [Select][+][-]
  1. b957a25f8f92eab1afe5968c5ee85b05303647edfcd6a6dd6b5eadcfdce5464c

Cheers,
Gus

* Greatest Of All Time
« Last Edit: March 22, 2024, 04:34:24 pm by Gustavo 'Gus' Carreno »
Lazarus 3.99(main) FPC 3.3.1(main) Ubuntu 23.10 64b Dark Theme
Lazarus 3.0.0(stable) FPC 3.2.2(stable) Ubuntu 23.10 64b Dark Theme
http://github.com/gcarreno

Gustavo 'Gus' Carreno

  • Hero Member
  • *****
  • Posts: 1132
  • Professional amateur ;-P
Re: Offical launch of the 1 Billion Row Challenge in Object Pascal
« Reply #32 on: March 24, 2024, 12:31:23 am »
Hey Y'all,

Have now added the hyperfine JSON files to the repo.
For anyone interested on your 10 runs.

Cheers,
Gus
Lazarus 3.99(main) FPC 3.3.1(main) Ubuntu 23.10 64b Dark Theme
Lazarus 3.0.0(stable) FPC 3.2.2(stable) Ubuntu 23.10 64b Dark Theme
http://github.com/gcarreno

Gustavo 'Gus' Carreno

  • Hero Member
  • *****
  • Posts: 1132
  • Professional amateur ;-P
Re: Offical launch of the 1 Billion Row Challenge in Object Pascal
« Reply #33 on: March 25, 2024, 04:37:08 pm »
Hey my challenge peeps,

Someone from the Telegrams has mentioned that our Rounding code is maybe flawed with regards to negative temps.
He's looking into this.
He's also hacking away at making the Delphi version of the baseline.
Be ready to jump on that PR button once we've come to the bottom of this.

Cheers,
Gus
Lazarus 3.99(main) FPC 3.3.1(main) Ubuntu 23.10 64b Dark Theme
Lazarus 3.0.0(stable) FPC 3.2.2(stable) Ubuntu 23.10 64b Dark Theme
http://github.com/gcarreno

Gustavo 'Gus' Carreno

  • Hero Member
  • *****
  • Posts: 1132
  • Professional amateur ;-P
Re: Offical launch of the 1 Billion Row Challenge in Object Pascal
« Reply #34 on: March 26, 2024, 12:32:21 am »
Hey Y'All,

After a bit of talking to a prospect entry person, I've come to the realisation that we had a very complicated rounding implementation.

The conclusion of that talk and further code is now the new official rounding code.

I've also altered the README.md file to include the code and the new SHA256 hash of the output, alongside an archived file containing the new baseline output.

Cheers,
Gus
Lazarus 3.99(main) FPC 3.3.1(main) Ubuntu 23.10 64b Dark Theme
Lazarus 3.0.0(stable) FPC 3.2.2(stable) Ubuntu 23.10 64b Dark Theme
http://github.com/gcarreno

Hartmut

  • Hero Member
  • *****
  • Posts: 768
Re: Offical launch of the 1 Billion Row Challenge in Object Pascal
« Reply #35 on: March 26, 2024, 07:22:38 pm »
There seems to be a bug in the new official rounding code:

Code: Pascal  [Select][+][-]
  1. function RoundExString(x: Double): String; inline;
  2. var
  3.   V, Q, R: Integer;
  4. begin
  5.   V := RoundExInteger(x);
  6.   if V < 0 then
  7.   begin
  8.     Result := '-';
  9.     V := -V;
  10.   end
  11.   else
  12.     Result := '';
  13.   Q := V div 10;
  14.   R := V - (Q * 10);
  15.   Result := IntToStr(Q) + '.' + IntToStr(R);
  16. end;

I think, the last line should be:
Code: Pascal  [Select][+][-]
  1. Result := Result + IntToStr(Q) + '.' + IntToStr(R);

Gustavo 'Gus' Carreno

  • Hero Member
  • *****
  • Posts: 1132
  • Professional amateur ;-P
Re: Offical launch of the 1 Billion Row Challenge in Object Pascal
« Reply #36 on: March 27, 2024, 05:51:51 am »
Hey Hartmut,

You are absolutely right!
Will correct ASAP!

Many thanks !!!

Cheers,
Gus
Lazarus 3.99(main) FPC 3.3.1(main) Ubuntu 23.10 64b Dark Theme
Lazarus 3.0.0(stable) FPC 3.2.2(stable) Ubuntu 23.10 64b Dark Theme
http://github.com/gcarreno

bytebites

  • Hero Member
  • *****
  • Posts: 647
Re: Offical launch of the 1 Billion Row Challenge in Object Pascal
« Reply #37 on: March 27, 2024, 10:26:07 am »
There seems to be a bug in the new official rounding code:

Code: Pascal  [Select][+][-]
  1. function RoundExString(x: Double): String; inline;
  2. var
  3.   V, Q, R: Integer;
  4. begin
  5.   V := RoundExInteger(x);
  6.   if V < 0 then
  7.   begin
  8.     Result := '-';
  9.     V := -V;
  10.   end
  11.   else
  12.     Result := '';
  13.   Q := V div 10;
  14.   R := V - (Q * 10);
  15.   Result := IntToStr(Q) + '.' + IntToStr(R);
  16. end;

I think, the last line should be:
Code: Pascal  [Select][+][-]
  1. Result := Result + IntToStr(Q) + '.' + IntToStr(R);

Which could be simplified to this:
Code: Pascal  [Select][+][-]
  1. function RoundExString(x: Double): String; inline;
  2. var
  3.   V, Q, R: Integer;
  4. begin
  5.   V := RoundExInteger(x);
  6.   divmod(V, 10, Q, R);
  7.   Result := IntToStr(Q) + '.' + chr(48 + Abs(R))
  8. end;
  9.    
« Last Edit: March 27, 2024, 05:32:15 pm by bytebites »

bytebites

  • Hero Member
  • *****
  • Posts: 647
Re: Offical launch of the 1 Billion Row Challenge in Object Pascal
« Reply #38 on: March 27, 2024, 10:40:49 am »
Another issue is that archived version of the baseline output contains 41000+ results, when 400 is expected and then shasum is perhaps wrong too,

paweld

  • Hero Member
  • *****
  • Posts: 1058
Re: Offical launch of the 1 Billion Row Challenge in Object Pascal
« Reply #39 on: March 27, 2024, 11:32:58 am »
generator allows you to generate a file with only 400 stations, you must use the -4 parameter.     
The checksum for such a file with 1 billion lines is: b957a25f8f92eab1afe5968c5ee85b05303647edfcd6a6dd6b5eadcfdce5464c
Best regards / Pozdrawiam
paweld

Gustavo 'Gus' Carreno

  • Hero Member
  • *****
  • Posts: 1132
  • Professional amateur ;-P
Re: Offical launch of the 1 Billion Row Challenge in Object Pascal
« Reply #40 on: March 27, 2024, 07:35:52 pm »
Hey bytebites,

Which could be simplified to this:
Code: Pascal  [Select][+][-]
  1. function RoundExString(x: Double): String; inline;
  2. var
  3.   V, Q, R: Integer;
  4. begin
  5.   V := RoundExInteger(x);
  6.   divmod(V, 10, Q, R);
  7.   Result := IntToStr(Q) + '.' + chr(48 + Abs(R))
  8. end;

That does seem quite more elegant !!!!

Thanks !!!

Will replace and give you credit ASAP!!

Cheers,
Gus
Lazarus 3.99(main) FPC 3.3.1(main) Ubuntu 23.10 64b Dark Theme
Lazarus 3.0.0(stable) FPC 3.2.2(stable) Ubuntu 23.10 64b Dark Theme
http://github.com/gcarreno

Gustavo 'Gus' Carreno

  • Hero Member
  • *****
  • Posts: 1132
  • Professional amateur ;-P
Re: Offical launch of the 1 Billion Row Challenge in Object Pascal
« Reply #41 on: March 27, 2024, 07:41:48 pm »
Hey bytebites,

Another issue is that archived version of the baseline output contains 41000+ results, when 400 is expected and then shasum is perhaps wrong too,

If you read the README.md in the Differences From Original section, you'll see that it was my personal choice to turn it up to 11.

Cheers,
Gus
Lazarus 3.99(main) FPC 3.3.1(main) Ubuntu 23.10 64b Dark Theme
Lazarus 3.0.0(stable) FPC 3.2.2(stable) Ubuntu 23.10 64b Dark Theme
http://github.com/gcarreno

Gustavo 'Gus' Carreno

  • Hero Member
  • *****
  • Posts: 1132
  • Professional amateur ;-P
Re: Offical launch of the 1 Billion Row Challenge in Object Pascal
« Reply #42 on: March 28, 2024, 08:21:29 pm »
Hey Y'All,

We seem to have a bit of a problem: https://github.com/gcarreno/1brc-ObjectPascal/tree/main/spelunking/roundex

See attached image.

Cheers,
Gus
Lazarus 3.99(main) FPC 3.3.1(main) Ubuntu 23.10 64b Dark Theme
Lazarus 3.0.0(stable) FPC 3.2.2(stable) Ubuntu 23.10 64b Dark Theme
http://github.com/gcarreno

Gustavo 'Gus' Carreno

  • Hero Member
  • *****
  • Posts: 1132
  • Professional amateur ;-P
Re: Offical launch of the 1 Billion Row Challenge in Object Pascal
« Reply #43 on: March 28, 2024, 11:51:31 pm »
 Hey Y'All,

Looks like there is a simple fix: Not using Double but Currency.

Code: Pascal  [Select][+][-]
  1. function RoundExDouble(x: Currency): Double;
  2. begin
  3.   Result := Ceil(x * 10) / 10;
  4. end;

Does not fix the issue that Delphi is not consistent with Double across Windows 32,64 and Linux 64.

But at least we now have consistency on our end!!!

Many thanks to @paweld for spotting the fix!!

Cheers,

Gus
Edited just now by Gustavo 'Gus' Carreno
Lazarus 3.99(main) FPC 3.3.1(main) Ubuntu 23.10 64b Dark Theme
Lazarus 3.0.0(stable) FPC 3.2.2(stable) Ubuntu 23.10 64b Dark Theme
http://github.com/gcarreno

Gustavo 'Gus' Carreno

  • Hero Member
  • *****
  • Posts: 1132
  • Professional amateur ;-P
Re: Offical launch of the 1 Billion Row Challenge in Object Pascal
« Reply #44 on: March 29, 2024, 05:53:09 am »
Hey Y'All,

With the rounding function changed, so did the output hash.
New hash and archived output from baseline are now up on the repo.

Cheers,

Gus
Lazarus 3.99(main) FPC 3.3.1(main) Ubuntu 23.10 64b Dark Theme
Lazarus 3.0.0(stable) FPC 3.2.2(stable) Ubuntu 23.10 64b Dark Theme
http://github.com/gcarreno

 

TinyPortal © 2005-2018