Recent

Author Topic: Offical launch of the 1 Billion Row Challenge in Object Pascal  (Read 21031 times)

Gustavo 'Gus' Carreno

  • Hero Member
  • *****
  • Posts: 1132
  • Professional amateur ;-P
Hey Y'All,

Official launch of the 1 Billion Row Challenge

Happy coding and don't forget to have fun!!

Cheers,
Gus
Lazarus 3.99(main) FPC 3.3.1(main) Ubuntu 23.10 64b Dark Theme
Lazarus 3.0.0(stable) FPC 3.2.2(stable) Ubuntu 23.10 64b Dark Theme
http://github.com/gcarreno

Gustavo 'Gus' Carreno

  • Hero Member
  • *****
  • Posts: 1132
  • Professional amateur ;-P
Re: Offical launch of the 1 Billion Row Challenge in Object Pascal
« Reply #1 on: March 12, 2024, 02:56:46 am »
Hey Y'All,

Just an FYI on the fact that the README.md file had a recent update that mentions:
  • The SHA256 hash of the input file.
  • The usage printout for the generator.

Cheers,
Gus
Lazarus 3.99(main) FPC 3.3.1(main) Ubuntu 23.10 64b Dark Theme
Lazarus 3.0.0(stable) FPC 3.2.2(stable) Ubuntu 23.10 64b Dark Theme
http://github.com/gcarreno

Hartmut

  • Hero Member
  • *****
  • Posts: 768
Re: Offical launch of the 1 Billion Row Challenge in Object Pascal
« Reply #2 on: March 12, 2024, 12:44:08 pm »
Hello Gus,

how will you proof, if the results of the competitors are correct?

a) There can be rounding-errors in the temparature mean values, depending on whether single or double or integer variables are used. What will you tolerate?

b) There can be sorting differences, because cities like "Abéché" are used. What will you tolerate?

Gustavo 'Gus' Carreno

  • Hero Member
  • *****
  • Posts: 1132
  • Professional amateur ;-P
Re: Offical launch of the 1 Billion Row Challenge in Object Pascal
« Reply #3 on: March 12, 2024, 03:11:43 pm »
Hey Hartmut,

Hello Gus,

how will you proof, if the results of the competitors are correct?

a) There can be rounding-errors in the temparature mean values, depending on whether single or double or integer variables are used. What will you tolerate?

b) There can be sorting differences, because cities like "Abéché" are used. What will you tolerate?

That's a very good question, and maybe something I have to address in the repo's readme file.

The piece of the puzzle that is missing is that we will base the testing with the output of what I'm calling the baseline version.

My brain has been acting up for the past days and I haven't been cotton free on it to actually produce that baseline.
But I'm getting some help from others on the Discord servers where we're coordinating this a bit more interactively.

The output of said baseline will then be run through a SHA256 tool that will produce a hash.
That hash will be the official hash to test against.
We already have included instruction for the 1 billion row file to test the hash and we also have the official hash value there.

This will be included in the readme once the baseline is finished and I wait, what I'm gonna guess, will be several hours. The objective of the baseline is to be single thread and use the worst container for the sorted list.

Hope this clears out your previous points.

Cheers,
Gus
Lazarus 3.99(main) FPC 3.3.1(main) Ubuntu 23.10 64b Dark Theme
Lazarus 3.0.0(stable) FPC 3.2.2(stable) Ubuntu 23.10 64b Dark Theme
http://github.com/gcarreno

Hartmut

  • Hero Member
  • *****
  • Posts: 768
Re: Offical launch of the 1 Billion Row Challenge in Object Pascal
« Reply #4 on: March 12, 2024, 05:22:10 pm »
Hello Gus, thanks for this info.

Gustavo 'Gus' Carreno

  • Hero Member
  • *****
  • Posts: 1132
  • Professional amateur ;-P
Re: Offical launch of the 1 Billion Row Challenge in Object Pascal
« Reply #5 on: March 12, 2024, 05:27:42 pm »
Hey Hartmut,

Hello Gus, thanks for this info.

You are more than welcome !!!

Cheers,
Gus
Lazarus 3.99(main) FPC 3.3.1(main) Ubuntu 23.10 64b Dark Theme
Lazarus 3.0.0(stable) FPC 3.2.2(stable) Ubuntu 23.10 64b Dark Theme
http://github.com/gcarreno

Thaddy

  • Hero Member
  • *****
  • Posts: 14825
  • Censorship about opinions does not belong here.
Re: Offical launch of the 1 Billion Row Challenge in Object Pascal
« Reply #6 on: March 12, 2024, 05:29:39 pm »
Gus, try the LLVM backend with linker optimization and see what happens....
bitrate is always calculated like this:sample rate * bitdepth * number of channels.

Gustavo 'Gus' Carreno

  • Hero Member
  • *****
  • Posts: 1132
  • Professional amateur ;-P
Re: Offical launch of the 1 Billion Row Challenge in Object Pascal
« Reply #7 on: March 12, 2024, 10:00:25 pm »
Hey Thaddy,

Gus, try the LLVM backend with linker optimization and see what happens....

Good point !!!
Will do that with the Delphi only entries !!
Haven't had one yet :D

The generator is now stable, at least the shared code, so not gonna touch that any more.

Cheers,
Gus
Lazarus 3.99(main) FPC 3.3.1(main) Ubuntu 23.10 64b Dark Theme
Lazarus 3.0.0(stable) FPC 3.2.2(stable) Ubuntu 23.10 64b Dark Theme
http://github.com/gcarreno

Gustavo 'Gus' Carreno

  • Hero Member
  • *****
  • Posts: 1132
  • Professional amateur ;-P
Re: Offical launch of the 1 Billion Row Challenge in Object Pascal
« Reply #8 on: March 17, 2024, 04:31:12 pm »
Hey Y'all,

There have been some doubts about the rules and rounding.

I've made some updates to the README.md file in order to rectify that.

Please have a look and if you still think that anything is not clear, please drop an issue on the repository.

Cheers,
Gus
Lazarus 3.99(main) FPC 3.3.1(main) Ubuntu 23.10 64b Dark Theme
Lazarus 3.0.0(stable) FPC 3.2.2(stable) Ubuntu 23.10 64b Dark Theme
http://github.com/gcarreno

Gustavo 'Gus' Carreno

  • Hero Member
  • *****
  • Posts: 1132
  • Professional amateur ;-P
Re: Offical launch of the 1 Billion Row Challenge in Object Pascal
« Reply #9 on: March 17, 2024, 09:27:30 pm »
Hey Y'All,

A long time ago someone mentioned the lack of a CONTRIBUTING.md file.

Took me a while to get around that, but one is now present.

I've also added templates for Bug and Feature Requests issues.

I've also included some code pertaining the official rounding algorithm for the output values.
Have a look at it in the Rounding section.

Cheers,
Gus
Lazarus 3.99(main) FPC 3.3.1(main) Ubuntu 23.10 64b Dark Theme
Lazarus 3.0.0(stable) FPC 3.2.2(stable) Ubuntu 23.10 64b Dark Theme
http://github.com/gcarreno

bytebites

  • Hero Member
  • *****
  • Posts: 647
Re: Offical launch of the 1 Billion Row Challenge in Object Pascal
« Reply #10 on: March 18, 2024, 02:03:54 pm »
Ceil function produced correct result.

Gustavo 'Gus' Carreno

  • Hero Member
  • *****
  • Posts: 1132
  • Professional amateur ;-P
Re: Offical launch of the 1 Billion Row Challenge in Object Pascal
« Reply #11 on: March 19, 2024, 08:24:36 am »
Hey bytebites,

Ceil function produced correct result.

I'm really sorry, you kinda lost me here.

Can you please elaborate?
Thanks!!

Cheers,
Gus
Lazarus 3.99(main) FPC 3.3.1(main) Ubuntu 23.10 64b Dark Theme
Lazarus 3.0.0(stable) FPC 3.2.2(stable) Ubuntu 23.10 64b Dark Theme
http://github.com/gcarreno

bytebites

  • Hero Member
  • *****
  • Posts: 647
Re: Offical launch of the 1 Billion Row Challenge in Object Pascal
« Reply #12 on: March 19, 2024, 08:30:14 am »
Math.Ceil gives same result than complicated PascalRound, at least sha256sum is same.

Thaddy

  • Hero Member
  • *****
  • Posts: 14825
  • Censorship about opinions does not belong here.
Re: Offical launch of the 1 Billion Row Challenge in Object Pascal
« Reply #13 on: March 19, 2024, 08:37:40 am »
You can also set the roundmode directly in fpc. I don't think that matters but there is a speed gain. There is also a speed gain using the LLVM back-end with linker optimization. And the others - the compiled languages - do that too.....
What means we are not testing the language, but the compiler..... Another reason to think twice.
And don't forget to inline and {$I-}. I expect yet another 10-15% gain with LLVM.
« Last Edit: March 19, 2024, 08:50:21 am by Thaddy »
bitrate is always calculated like this:sample rate * bitdepth * number of channels.

Gustavo 'Gus' Carreno

  • Hero Member
  • *****
  • Posts: 1132
  • Professional amateur ;-P
Re: Offical launch of the 1 Billion Row Challenge in Object Pascal
« Reply #14 on: March 19, 2024, 03:46:12 pm »
Hey bytebites,

Math.Ceil gives same result than complicated PascalRound, at least sha256sum is same.

That is awesome and I thank you very, very much!!!

This will most certainly shut up a certain person's whining !!

Cheers,
Gus
Lazarus 3.99(main) FPC 3.3.1(main) Ubuntu 23.10 64b Dark Theme
Lazarus 3.0.0(stable) FPC 3.2.2(stable) Ubuntu 23.10 64b Dark Theme
http://github.com/gcarreno

 

TinyPortal © 2005-2018