Lazarus
Installation => General => Topic started by: JD on September 10, 2011, 10:55:14 am
-
Hi everyone,
I just read this article on Beeography "Which Programming Language is the best?" and according to him, the FPC compiler is no longer the best optimized open source compiler.
http://beeography.wordpress.com/2011/05/04/which-programming-language-is-best/#more-483
In the study, it was even demonstrated that the FPC compiler is slower than the Java 6 server! :o How can that be possible?
Does anyone have a contrary opinion and/or articles that proves that this is not actually the case?
JD
-
In the study, it was even demonstrated that the FPC compiler is slower than the Java 6 server! How can that be possible?
Actually, the benchmark game has a number of parameters which if you set to some certain values, could make a language down or up drastically as well as others. It's not a reliable source. If there's also FPC server entry, that would be fairer.
Does anyone have a contrary opinion and/or articles that proves that this is not actually the case?
Why don't you try yourself? ;)
* hope you have enough time (and your computer has enough resources) waiting for Java programs to complete.
From scientific point of view, (bytecode) interpreted programs would NEVER exceed the speed of compiled programs, if the optimizations applied are about the same weight, even when JIT-ed. Because JIT-ing takes (a long) time + JIT result is usually not flushed into disk but retained in memory for fast access, but OS may swap the pages any time, causing a lot of page faults and slows the program down, a lot.
-
AFAIK, benchmark game does not compare exactly the same programs written in different languages. The programs may be very different (based on different algorithms or using differently multi-threading).
-
In the study, it was even demonstrated that the FPC compiler is slower than the Java 6 server! :o How can that be possible?
1) Choice of benchmarks,
2) allowing to tweak the runtime in a way an user usually doesn't
3) the fact that the FPC community mostly got fed up with the shootout and stopped updating the benchmarks.
Does anyone have a contrary opinion and/or articles that proves that this is not actuallt the case?
Basically I think only the magnitude matters. (<10, 10-50 >50), the rest is basically noise.
-
This brings two popular idioms to my mind: 1) Benchmarking apples against oranges, and 2) bytecode has snowball's chance in hell to outperform native code ;)
-
This brings two popular idioms to my mind: 1) Benchmarking apples against oranges, and 2) bytecode has snowball's chance in hell to outperform native code ;)
Instead of comparing apples and oranges, an even better analogy is comparing apples with apple trees.
OPascal is a language you create a "Java language" with. I have several articles bookmarked about that very subject "creating compilers with delphi". This series of articles has you building COMPILERS exactly like the "Java compiler" they are benching against,using Delphi!
So how in the world are they even in the same category?
Would you use you "Java compiler" to construct a new, custom compiler?
No, you would use the language for building compilers. Assembly,Opascal, or C.
The beauty of using OPascal is you get true object oriented code at the deepest genetic level possible. You have all the benefits of OP for low level tasks (like compilers) as you do for gui apps. And remember this, we ASSUME that the OP model comes with overhead, hence slower, than say, Assembly or C. This is actually not so much the case at all because the object model was incorporated into the compiler itself. So OPascal is like an "Assembly driver" almost it seems and very closely coupled with the underlying Assembly itself vs. your Java compiler which could have been WRITTEN in Opascal.
So basically OPascal stands alone in many ways...Still...and always has since TurboPascal days
-
Would you use you "Java compiler" to construct a new, custom compiler?
Unfortunately, this is the case in many compiler classes, even they use JLex and JavaCUP for scanner and parser construction. Yucks!
And remember this, we ASSUME that the OP model comes with overhead, hence slower, than say, Assembly or C
A little slower execution, but a much faster development (say, 3-20 times), and much shorter code.
-
Oh well, as Bee has stated on the replies part, the test was on 2.4.0. But then when they re-tested it using 2.4.2, it was NO 1 once again!
-
But then when they re-tested it using 2.4.2, it was NO 1 once again!
Is there some link to confirm this?
-
But then when they re-tested it using 2.4.2, it was NO 1 once again!
Is there some link to confirm this?
When you follow the link from the first post, then click on the grid showing the languages order, you get to this page:
http://shootout.alioth.debian.org/u32q/which-language-is-best.php?calc=chart&gcc=on&gpp=on&fpascal=on&java=on&csharp=on&xfullcpu=1&xmem=1&xloc=0&nbody=1&fannkuchredux=1&meteor=1&fasta=1&spectralnorm=1&revcomp=1&mandelbrot=1&knucleotide=1®exdna=0&pidigits=1&chameneosredux=1&threadring=1&binarytrees=1
It seems that no one here read the comments from the page which is linked in first post. ;)
-
@Zoran: Thanks, you made me pleased.
Some time ago I looked to these benchmarks and I noticed that C++ is faster but consumes much more memory (2-3 times in many cases). Compilation speed of Pascal compilers is usually very good. This link compares all.
-
I just read the link in the fist post and to me one thing is quite obvious... it is FreePascal that all the others are attempting to match.
It is Freepascal that is on top. You can tell by the methods they use to try to knock it down.
FreePascal is "genetically" superior. It has advantages deep at the core level that not even C can boast. Only thing C can boast is in the speed department. Even that is misleading because you can easily optimize your FreePascal code for speed. These tests do not reflect performance based upon OPTIMIZED FreePascal code.
FreePascal has OO built into the compiler itself. all the other players are trying to emulate OO after the fact. TurboPascal took an approach waaaayyy back long ago that set it in a class on it's own. When everybody kept the "linear model" TurboPascal went full blast into OO with it's compiler, down at the genetic level. To be able to compete at this level, to even be on the playing field, would require that even C needs to go "back to the drawing board".
... FROM SCRATCH
All these "compiler shootouts" are moot because no other language took the OO route into their compiler.
-
I like Lazarus FPC, with Qt takes a lot of time to compile a Hello World application.
-
Study the test closer and the truth according to the numbers is that FPC is a good 20% FASTER than C!
Oops!
Edit:
20% size reduction
-
@Zoran:
As most people are lazy, they just click and accept what's presented by default in that page. The benchmark only compares speed by default, ignoring memory and code size. That's why C/C++ wins. When you add the other two, bye2 C/C++...
AFAIU, if you tick all 3 measurements, it means that the top language implementations have the best balance of the 3.
-
But then when they re-tested it using 2.4.2, it was NO 1 once again!
Is there some link to confirm this?
When you follow the link from the first post, then click on the grid showing the languages order, you get to this page:
http://shootout.alioth.debian.org/u32q/which-language-is-best.php?calc=chart&gcc=on&gpp=on&fpascal=on&java=on&csharp=on&xfullcpu=1&xmem=1&xloc=0&nbody=1&fannkuchredux=1&meteor=1&fasta=1&spectralnorm=1&revcomp=1&mandelbrot=1&knucleotide=1®exdna=0&pidigits=1&chameneosredux=1&threadring=1&binarytrees=1
It seems that no one here read the comments from the page which is linked in first post. ;)
I have followed the links on the top of page from first post, and I have even tried to open dozen of links from the archives on the right. Unfortunately without success, so I gave up and therefore asked for a link. Silly me, it didn't come to my mind something so simple as hovering mouse over image.
-
AFAIK, benchmark game does not compare exactly the same programs written in different languages. The programs may be very different (based on different algorithms or using differently multi-threading).
Can "exactly the same programs" always be written in different languages?
-
From scientific point of view, (bytecode) interpreted programs would NEVER exceed the speed of compiled programs, if the optimizations applied are about the same weight, even when JIT-ed. Because JIT-ing takes (a long) time + JIT result is usually not flushed into disk but retained in memory for fast access, but OS may swap the pages any time, causing a lot of page faults and slows the program down, a lot.
This Free Pascal n-body program took 34.96 secs elapsed time with workload n=50,000,000 http://shootout.alioth.debian.org/u64q/program.php?test=nbody&lang=fpascal&id=1 (http://shootout.alioth.debian.org/u64q/program.php?test=nbody&lang=fpascal&id=1)
This Java n-body program took 22.49 secs elapsed time with the same workload http://shootout.alioth.debian.org/u64q/program.php?test=nbody&lang=java&id=2 (http://shootout.alioth.debian.org/u64q/program.php?test=nbody&lang=java&id=2)
From scientific point of view, the Java program was faster.
-
bytecode has snowball's chance in hell to outperform native code
Since J2SE 1.3 (May 8, 2000) the default HotSpot Java VM has compiled bytecode to native code.
Remember how HotSpot works. It starts by running your program with an interpreter. When it discovers that some method is "hot" -- that is, executed a lot, either because it is called a lot or because it contains loops that loop a lot -- it sends that method off to be compiled. After that one of two things will happen, either the next time the method is called the compiled version will be invoked (instead of the interpreted version) or the currently long running loop will be replaced, while still running, with the compiled method. The latter is known as "on stack replacement", or OSR.
-
Can "exactly the same programs" always be written in different languages?
Yes, if all languages agree not to use their libraries and/or managed types (e.g. strings).
From scientific point of view, the Java program was faster.
No, that's practical point of view. As already said before, the implementation of the program differs. The data structure, algorithms behind it, implemented algorithms for the problem itself, etc.
Since J2SE 1.3 (May 8, 2000) the default HotSpot Java VM has compiled bytecode to native code
Doesn't mean it's faster than native code. JIT-ing takes time (and memory, and huge memory usage in turns causes slow down due to page faults), and that's even the actual bottleneck of bytecode interpreted implementations.
-
Can "exactly the same programs" always be written in different languages?
)
Some benchmarks like "thread-ring" are implemented differently. Others (like "mandelbrot") seems to be very similar.
-
Can "exactly the same programs" always be written in different languages?
)Some benchmarks like "thread-ring" are implemented differently. Others (like "mandelbrot") seems to be very similar.
Yes, one of the more interesting differences between programming languages is the way they deal with concurrency and that's why thread-ring was originally shown - but once the measurements started to be made on a quad-core machine it turned out that there were naively parallel approaches to some of the other tasks (and somewhat comically although the thread-ring task uses a lot of threads it's actually sequential).
Rather than "exactly the same programs" there's some attempt "to be very similar" in that the programs should implement the same algorithm.
-
As already said before, the implementation of the program differs. The data structure, algorithms behind it, implemented algorithms for the problem itself, etc.
The Pascal program uses an array of records and the Java program uses an array of objects - we can all see that the Pascal n-body program is line-by-line very similar to the Java n-body program.
Please tell us exactly difference you see between the source code of this FreePascal n-body program
http://shootout.alioth.debian.org/u64q/program.php?test=nbody&lang=fpascal&id=1 (http://shootout.alioth.debian.org/u64q/program.php?test=nbody&lang=fpascal&id=1)
and the source code of this Java n-body program
http://shootout.alioth.debian.org/u64q/program.php?test=nbody&lang=java&id=2 (http://shootout.alioth.debian.org/u64q/program.php?test=nbody&lang=java&id=2)
that you say accounts for the Java program running so much faster.
Since J2SE 1.3 (May 8, 2000) the default HotSpot Java VM has compiled bytecode to native code
Doesn't mean it's faster than native code. JIT-ing takes time (and memory, and huge memory usage in turns causes slow down due to page faults), and that's even the actual bottleneck of bytecode interpreted implementations.
Bytecode loading, and JITing does take time, but so little time that even when those n-body programs are run just for a few tenths of a second - the Pascal program is only faster by 1/100th of a second.
-
The Pascal program uses global variable while the Java version uses local variable for the body variable, which is accessed very frequently. I remember a discussion about it in the mailing list, that FPC can't yet optimize global variable access. AFAIR, this exists before 2.4.4 and I guess it's not yet improved since. The Java page clearly states that they drop the first execution, where the JIT should normally happen (subsequent executions reuse what's already in memory AFAIK).
-
The Java page clearly states that they drop the first execution...
That is not true. Here is the "Java page" that shows the n-body program 22.49 secs elapsed time and there's nothing about "drop the first execution" -
http://shootout.alioth.debian.org/u64q/java.php (http://shootout.alioth.debian.org/u64q/java.php)
As stated on the Help page - "Time measurements include program startup time".
That 22.49 secs includes everything that happened - bytecode loading and JITing - from just before the program was run with
/usr/local/src/jdk1.7.0/bin/java -server -XX:+TieredCompilation -XX:+AggressiveOpts nbody 50000000
until after the Java program ends.
-
1) Choice of benchmarks
Are you simply stating the obvious, that these are measurements of specific tasks; or do you mean some innuendo, suggesting tasks were chosen so the FreePascal programs would not be as fast as the Java programs?
2) allowing to tweak the runtime in a way an user usually doesn't
Allowed for Free Pascal, as well as Java.
3) the fact that the FPC community mostly got fed up with the shootout and stopped updating the benchmarks.
There have not been a lot more Java programs contributed than Free Pascal programs contributed, so perhaps the FPC community put extra effort into these programs before they "got fed up" -
http://shootout.alioth.debian.org/dont-jump-to-conclusions.php#effort (http://shootout.alioth.debian.org/dont-jump-to-conclusions.php#effort)
-
In the study, it was even demonstrated that the FPC compiler is slower than the Java 6 server! :o How can that be possible?
Please explain what you think this means - "the FPC compiler is slower than the Java 6 server".
Why do you doubt that it's possible?