Recent

Author Topic: Free Pascal compiler no longer the best optimized open source compiler  (Read 28389 times)

avra

  • Hero Member
  • *****
  • Posts: 1737
    • Additional info
Re: Free Pascal compiler no longer the best optimized open source compiler
« Reply #15 on: September 14, 2011, 12:08:09 pm »
But then when they re-tested it using 2.4.2, it was NO 1 once again!
Is there some link to confirm this?

When you follow the link from the first post, then click on the grid showing the languages order, you get to this page:

http://shootout.alioth.debian.org/u32q/which-language-is-best.php?calc=chart&gcc=on&gpp=on&fpascal=on&java=on&csharp=on&xfullcpu=1&xmem=1&xloc=0&nbody=1&fannkuchredux=1&meteor=1&fasta=1&spectralnorm=1&revcomp=1&mandelbrot=1&knucleotide=1&regexdna=0&pidigits=1&chameneosredux=1&threadring=1&binarytrees=1

It seems that no one here read the comments from the page which is linked in first post. ;)

I have followed the links on the top of page from first post, and I have even tried to open dozen of links from the archives on the right. Unfortunately without success, so I gave up and therefore asked for a link. Silly me, it didn't come to my mind something so simple as hovering mouse over image.
ct2laz - Conversion between Lazarus and CodeTyphon
bithelpers - Bit manipulation for standard types
pasettimino - Siemens S7 PLC lib

igouy

  • New Member
  • *
  • Posts: 25
Re: Free Pascal compiler no longer the best optimized open source compiler
« Reply #16 on: September 15, 2011, 01:11:34 am »
AFAIK, benchmark game does not compare exactly the same programs written in different languages. The programs may be very different (based on different algorithms or using differently multi-threading).
Can "exactly the same programs" always be written in different languages?

igouy

  • New Member
  • *
  • Posts: 25
Re: Free Pascal compiler no longer the best optimized open source compiler
« Reply #17 on: September 15, 2011, 01:23:15 am »
From scientific point of view, (bytecode) interpreted programs would NEVER exceed the speed of compiled programs, if the optimizations applied are about the same weight, even when JIT-ed. Because JIT-ing takes (a long) time + JIT result is usually not flushed into disk but retained in memory for fast access, but OS may swap the pages any time, causing a lot of page faults and slows the program down, a lot.

This Free Pascal n-body program took 34.96 secs elapsed time with workload n=50,000,000 http://shootout.alioth.debian.org/u64q/program.php?test=nbody&lang=fpascal&id=1

This Java n-body program took 22.49 secs elapsed time with the same workload http://shootout.alioth.debian.org/u64q/program.php?test=nbody&lang=java&id=2

From scientific point of view, the Java program was faster.

igouy

  • New Member
  • *
  • Posts: 25
Re: Free Pascal compiler no longer the best optimized open source compiler
« Reply #18 on: September 15, 2011, 01:33:38 am »
bytecode has snowball's chance in hell to outperform native code

Since J2SE 1.3 (May 8, 2000) the default HotSpot Java VM has compiled bytecode to native code.

Quote
Remember how HotSpot works. It starts by running your program with an interpreter. When it discovers that some method is "hot" -- that is, executed a lot, either because it is called a lot or because it contains loops that loop a lot -- it sends that method off to be compiled. After that one of two things will happen, either the next time the method is called the compiled version will be invoked (instead of the interpreted version) or the currently long running loop will be replaced, while still running, with the compiled method. The latter is known as "on stack replacement", or OSR.

Leledumbo

  • Hero Member
  • *****
  • Posts: 8114
  • Programming + Glam Metal + Tae Kwon Do = Me
Re: Free Pascal compiler no longer the best optimized open source compiler
« Reply #19 on: September 15, 2011, 01:41:07 am »
Quote
Can "exactly the same programs" always be written in different languages?
Yes, if all languages agree not to use their libraries and/or managed types (e.g. strings).
Quote
From scientific point of view, the Java program was faster.
No, that's practical point of view. As already said before, the implementation of the program differs. The data structure, algorithms behind it, implemented algorithms for the problem itself, etc.
Quote
Since J2SE 1.3 (May 8, 2000) the default HotSpot Java VM has compiled bytecode to native code
Doesn't mean it's faster than native code. JIT-ing takes time (and memory, and huge memory usage in turns causes slow down due to page faults), and that's even the actual bottleneck of bytecode interpreted implementations.

Blaazen

  • Hero Member
  • *****
  • Posts: 2782
  • POKE 54296,15
    • Eye-Candy Controls
Re: Free Pascal compiler no longer the best optimized open source compiler
« Reply #20 on: September 15, 2011, 01:56:09 am »
Quote
Can "exactly the same programs" always be written in different languages?
)

Some benchmarks like "thread-ring" are implemented differently. Others (like "mandelbrot") seems to be very similar.
Lazarus 2.1.0 r61214:62238 FPC 3.3.1 r40507 x86_64-linux-qt Chakra, Qt 4.8.7/5.13.2, Plasma 5.17.3
Lazarus 1.8.2 r57369 FPC 3.0.4 i386-win32-win32/win64 Wine 3.21

Try Eye-Candy Controls: https://sourceforge.net/projects/eccontrols/files/

igouy

  • New Member
  • *
  • Posts: 25
Re: Free Pascal compiler no longer the best optimized open source compiler
« Reply #21 on: September 15, 2011, 03:07:44 am »
Quote
Can "exactly the same programs" always be written in different languages?
)Some benchmarks like "thread-ring" are implemented differently. Others (like "mandelbrot") seems to be very similar.

Yes, one of the more interesting differences between programming languages is the way they deal with concurrency and that's why thread-ring was originally shown - but once the measurements started to be made on a quad-core machine it turned out that there were naively parallel approaches to some of the other tasks (and somewhat comically although the thread-ring task uses a lot of threads it's actually sequential).

Rather than "exactly the same programs" there's some attempt "to be very similar" in that the programs should implement the same algorithm.

igouy

  • New Member
  • *
  • Posts: 25
Re: Free Pascal compiler no longer the best optimized open source compiler
« Reply #22 on: September 15, 2011, 03:24:55 am »
As already said before, the implementation of the program differs. The data structure, algorithms behind it, implemented algorithms for the problem itself, etc.
The Pascal program uses an array of records and the Java program uses an array of objects - we can all see that the Pascal n-body program is line-by-line very similar to the Java n-body program.

Please tell us exactly difference you see between the source code of this FreePascal n-body program

http://shootout.alioth.debian.org/u64q/program.php?test=nbody&lang=fpascal&id=1

and the source code of this Java n-body program

http://shootout.alioth.debian.org/u64q/program.php?test=nbody&lang=java&id=2

that you say accounts for the Java program running so much faster.


Quote
Since J2SE 1.3 (May 8, 2000) the default HotSpot Java VM has compiled bytecode to native code
Doesn't mean it's faster than native code. JIT-ing takes time (and memory, and huge memory usage in turns causes slow down due to page faults), and that's even the actual bottleneck of bytecode interpreted implementations.

Bytecode loading, and JITing does take time, but so little time that even when those n-body programs are run just for a few tenths of a second - the Pascal program is only faster by 1/100th of a second.

Leledumbo

  • Hero Member
  • *****
  • Posts: 8114
  • Programming + Glam Metal + Tae Kwon Do = Me
Re: Free Pascal compiler no longer the best optimized open source compiler
« Reply #23 on: September 15, 2011, 08:11:36 am »
The Pascal program uses global variable while the Java version uses local variable for the body variable, which is accessed very frequently. I remember a discussion about it in the mailing list, that FPC can't yet  optimize global variable access. AFAIR, this exists before 2.4.4 and I guess it's not yet improved since. The Java page clearly states that they drop the first execution, where the JIT should normally happen (subsequent executions reuse what's already in memory AFAIK).

igouy

  • New Member
  • *
  • Posts: 25
Re: Free Pascal compiler no longer the best optimized open source compiler
« Reply #24 on: September 15, 2011, 09:38:44 am »
The Java page clearly states that they drop the first execution...
That is not true. Here is the "Java page" that shows the n-body program 22.49 secs elapsed time and there's nothing about "drop the first execution" -

http://shootout.alioth.debian.org/u64q/java.php

As stated on the Help page - "Time measurements include program startup time".

That 22.49 secs includes everything that happened - bytecode loading and JITing - from just before the program was run with

/usr/local/src/jdk1.7.0/bin/java -server -XX:+TieredCompilation -XX:+AggressiveOpts nbody 50000000

until after the Java program ends.

igouy

  • New Member
  • *
  • Posts: 25
Re: Free Pascal compiler no longer the best optimized open source compiler
« Reply #25 on: September 15, 2011, 10:09:09 am »
1) Choice of benchmarks
Are you simply stating the obvious, that these are measurements of specific tasks; or do you mean some innuendo, suggesting tasks were chosen so the FreePascal programs would not be as fast as the Java programs?

Quote
2) allowing to tweak the runtime in a way an user usually doesn't
Allowed for Free Pascal, as well as Java.

Quote
3) the fact that the FPC community mostly got fed up with the shootout and stopped updating the benchmarks.
There have not been a lot more Java programs contributed than Free Pascal programs contributed, so perhaps the FPC community put extra effort into these programs before they "got fed up" -

http://shootout.alioth.debian.org/dont-jump-to-conclusions.php#effort
« Last Edit: September 15, 2011, 10:17:41 am by igouy »

igouy

  • New Member
  • *
  • Posts: 25
Re: Free Pascal compiler no longer the best optimized open source compiler
« Reply #26 on: September 15, 2011, 10:16:52 am »
In the study, it was even demonstrated that the FPC compiler is slower than the Java 6 server!  :o How can that be possible?

Please explain what you think this means - "the FPC compiler is slower than the Java 6 server".

Why do you doubt that it's possible?