Free Pascal compiler no longer the best optimized open source compiler

[avra]

But then when they re-tested it using 2.4.2, it was NO 1 once again!

Is there some link to confirm this?

When you follow the link from the first post, then click on the grid showing the languages order, you get to this page:

http://shootout.alioth.debian.org/u32q/which-language-is-best.php?calc=chart&gcc=on&gpp=on&fpascal=on&java=on&csharp=on&xfullcpu=1&xmem=1&xloc=0&nbody=1&fannkuchredux=1&meteor=1&fasta=1&spectralnorm=1&revcomp=1&mandelbrot=1&knucleotide=1&regexdna=0&pidigits=1&chameneosredux=1&threadring=1&binarytrees=1

It seems that no one here read the comments from the page which is linked in first post.

I have followed the links on the top of page from first post, and I have even tried to open dozen of links from the archives on the right. Unfortunately without success, so I gave up and therefore asked for a link. Silly me, it didn't come to my mind something so simple as hovering mouse over image.

[igouy]

AFAIK, benchmark game does not compare exactly the same programs written in different languages. The programs may be very different (based on different algorithms or using differently multi-threading).

Can "exactly the same programs" always be written in different languages?

[igouy]

From scientific point of view, (bytecode) interpreted programs would NEVER exceed the speed of compiled programs, if the optimizations applied are about the same weight, even when JIT-ed. Because JIT-ing takes (a long) time + JIT result is usually not flushed into disk but retained in memory for fast access, but OS may swap the pages any time, causing a lot of page faults and slows the program down, a lot.

This Free Pascal n-body program took 34.96 secs elapsed time with workload n=50,000,000 http://shootout.alioth.debian.org/u64q/program.php?test=nbody&lang=fpascal&id=1

This Java n-body program took 22.49 secs elapsed time with the same workload http://shootout.alioth.debian.org/u64q/program.php?test=nbody&lang=java&id=2

From scientific point of view, the Java program was faster.

[igouy]

bytecode has snowball's chance in hell to outperform native code

Since J2SE 1.3 (May 8, 2000) the default HotSpot Java VM has compiled bytecode to native code.

Remember how HotSpot works. It starts by running your program with an interpreter. When it discovers that some method is "hot" -- that is, executed a lot, either because it is called a lot or because it contains loops that loop a lot -- it sends that method off to be compiled. After that one of two things will happen, either the next time the method is called the compiled version will be invoked (instead of the interpreted version) or the currently long running loop will be replaced, while still running, with the compiled method. The latter is known as "on stack replacement", or OSR.

[Leledumbo]

Can "exactly the same programs" always be written in different languages?

Yes, if all languages agree not to use their libraries and/or managed types (e.g. strings).

From scientific point of view, the Java program was faster.

No, that's practical point of view. As already said before, the implementation of the program differs. The data structure, algorithms behind it, implemented algorithms for the problem itself, etc.

Since J2SE 1.3 (May 8, 2000) the default HotSpot Java VM has compiled bytecode to native code

Doesn't mean it's faster than native code. JIT-ing takes time (and memory, and huge memory usage in turns causes slow down due to page faults), and that's even the actual bottleneck of bytecode interpreted implementations.

[Blaazen]

Can "exactly the same programs" always be written in different languages?

)

Some benchmarks like "thread-ring" are implemented differently. Others (like "mandelbrot") seems to be very similar.

[igouy]

Can "exactly the same programs" always be written in different languages?

)Some benchmarks like "thread-ring" are implemented differently. Others (like "mandelbrot") seems to be very similar.

Yes, one of the more interesting differences between programming languages is the way they deal with concurrency and that's why thread-ring was originally shown - but once the measurements started to be made on a quad-core machine it turned out that there were naively parallel approaches to some of the other tasks (and somewhat comically although the thread-ring task uses a lot of threads it's actually sequential).

Rather than "exactly the same programs" there's some attempt "to be very similar" in that the programs should implement the same algorithm.

[igouy]

As already said before, the implementation of the program differs. The data structure, algorithms behind it, implemented algorithms for the problem itself, etc.

The Pascal program uses an array of records and the Java program uses an array of objects - we can all see that the Pascal n-body program is line-by-line very similar to the Java n-body program.

Please tell us exactly difference you see between the source code of this FreePascal n-body program

http://shootout.alioth.debian.org/u64q/program.php?test=nbody&lang=fpascal&id=1

and the source code of this Java n-body program

http://shootout.alioth.debian.org/u64q/program.php?test=nbody&lang=java&id=2

that you say accounts for the Java program running so much faster.

Since J2SE 1.3 (May 8, 2000) the default HotSpot Java VM has compiled bytecode to native code

Doesn't mean it's faster than native code. JIT-ing takes time (and memory, and huge memory usage in turns causes slow down due to page faults), and that's even the actual bottleneck of bytecode interpreted implementations.

Bytecode loading, and JITing does take time, but so little time that even when those n-body programs are run just for a few tenths of a second - the Pascal program is only faster by 1/100th of a second.

[Leledumbo]

The Pascal program uses global variable while the Java version uses local variable for the body variable, which is accessed very frequently. I remember a discussion about it in the mailing list, that FPC can't yet  optimize global variable access. AFAIR, this exists before 2.4.4 and I guess it's not yet improved since. The Java page clearly states that they drop the first execution, where the JIT should normally happen (subsequent executions reuse what's already in memory AFAIK).

[igouy]

The Java page clearly states that they drop the first execution...

That is not true. Here is the "Java page" that shows the n-body program 22.49 secs elapsed time and there's nothing about "drop the first execution" -

http://shootout.alioth.debian.org/u64q/java.php

As stated on the Help page - "Time measurements include program startup time".

That 22.49 secs includes everything that happened - bytecode loading and JITing - from just before the program was run with

/usr/local/src/jdk1.7.0/bin/java -server -XX:+TieredCompilation -XX:+AggressiveOpts nbody 50000000

until after the Java program ends.

[igouy]

1) Choice of benchmarks

Are you simply stating the obvious, that these are measurements of specific tasks; or do you mean some innuendo, suggesting tasks were chosen so the FreePascal programs would not be as fast as the Java programs?

2) allowing to tweak the runtime in a way an user usually doesn't

Allowed for Free Pascal, as well as Java.

3) the fact that the FPC community mostly got fed up with the shootout and stopped updating the benchmarks.

There have not been a lot more Java programs contributed than Free Pascal programs contributed, so perhaps the FPC community put extra effort into these programs before they "got fed up" -

http://shootout.alioth.debian.org/dont-jump-to-conclusions.php#effort

[igouy]

In the study, it was even demonstrated that the FPC compiler is slower than the Java 6 server!  How can that be possible?

Please explain what you think this means - "the FPC compiler is slower than the Java 6 server".

Why do you doubt that it's possible?