Checking Speed of SynEdit Highlighters

[Morton]

IMHO optimization should follow the project/package settings.

IMHO well tested units, especially high performance ones, should use the highest save optimization
and not depend on project/package settings.

[taazz]

IMHO optimization should follow the project/package settings.

IMHO well tested units, especially high performance ones, should use the highest save optimization
and not depend on project/package settings.

as long as it doesn't interfere with debug info, stepping into and variable watch I agree that it is imperative to add the needed optimizations.

[Martin_fr]

IMHO the question is, if the entire package should be shipped -O2

Or better: If it should follow the project on this. So buildmodes would take care of everything.

It is possible to force buildmode settings from the project to the package, but it needs to be set up in the project, and it is not as trivial as it should be.

[taazz]

optimizations that do not interfere with debugging can be set at the unit level otherwise they should always follow the projects settings (not the package all the way down to fpc packages), that will make things easier for settings like optimization, dynamic linking etc.

[Edson]

After some tests I have new comparisons of speed. Now I have choosen the Lazarus Perl Highlighter:

* Perl highlighter (hash-functions: 238 functions of 2167):  1.84seg 1.82seg 1.83seg
* Perl highlighter (with first-char-prefix: 26 functions):    1.75seg 1.76seg 1.75seg

Now I have taken the Perl highlighter (hash) and just replaced the hash algorithm to the prefix algorithm, keeping the others functions. So the comparison is very fair.

I hope someone could verify my results.

Attached: the prefix-highlighter used and the perl file of test.

[Morton]

Edson you could try another version with a case table instead of the call table.
Maybe casetables allow inlining.
Ordering your "if" cascades by statistical frequency may help to.
Some of your "if" cascades could likely be improved by a case table.

To get the last bit of of performance:
With any compiler I have ever used it may sometimes be helpful to insert a never used method to improve alignment.

[Morton]

Edson with level two or three optimization yours is slower.
As I said the hashed call tables works best with high optimization.
Linux 64Bit.

[Morton]

Edson your Perl HL produces a different amout of tokens than the one included in the newest Lazarus.

To all:

I have replaced the "fIdentFuncTable[HashKey]" with a "Case HashKey of"
in the Pascal highlighter.

Newer FPC compilers have changed nothing.
in contrary to Delphi:
"fIdentFuncTable[HashKey]"
has no significant speed advantage to
calling the methods with a "Case HashKey of"

[Martin_fr]

Not sure if fpc optimizes "case of" by using jump tables, or if they always are cascaded "if". Could find out with -al , looking at the asm generated.


Have you tried adding "nostackframe;" to the "func123" procedures?

[Edson]

Thanks Morton for checking my results.

Edson you could try another version with a case table instead of the call table.
Maybe casetables allow inlining.
Ordering your "if" cascades by statistical frequency may help to.
Some of your "if" cascades could likely be improved by a case table.

OK.
Probably this is not a fair comparison, because the prefix-algorithm is bad implemented. I just wanted to show to even in this way, the prefix algorithm could be faster. I'm sure that including just one more case table, it will be faster.

Edson your Perl HL produces a different amout of tokens than the one included in the newest Lazarus.

I'm comparing with the Perl Highlighter of Lazarus 1.0.12. I checked the tokens for a small sample, and it was the same.
Moreover, this prefix-highlighter is just for testing, no for production.

I'll try to do a better comparison.

PD. I haven't compared the Pascal Highlighter, because it has foldind implemented.

[Morton]

nostackframe above level 1 optimization causes access violations with call tables but not
with Case tables.

No noticeable speed effect.
Tested with Edsons Perl HL which has not so much methods.

[Morton]

I just wanted to show to even in this way, the prefix algorithm could be faster.

As noted in my first post: It may depend on the compiler what works best.

[Edson]

Just adding a simple case and interchanging one if (like Morton suggested), these are  the new results:

* Perl highlighter (hash-functions: 238 functions of 2167):  1.84seg 1.82seg 1.83seg
* Perl highlighter (with first-char-prefix: 26 functions):    1.68seg 1.68seg 1.68seg

And this is, having still, a bad implementation of the prefix algorithm. It could be easily improved.

I really reconsider using the hash-algorithm for highlighters. IMHO, this is an issue of algorithm more than of compiler.

[Morton]

IMHO, this is an issue of algorithm more than of compiler.

Some things will not work with another compiler or not as expected.

BTW the fastest possible Algorithm would be to use a multi dimensional matrix.
The length of the longest key gives the number of required dimensions.
The required amount of memory is prohibitive.
A sparse version implemented with dynamic arrays may be feasible on todays PCs.
with many GB of RAM.
Trees are much slower and became obsolete with 32Bit.

[Morton]

The fastest highlighter ever for Delphi: SynUniHighlighter by Fantasist.
Partly it was so fast because it leaked memory like hell in old versions.
Maybe fixed in newer versions.

It is criptable.
There have been hundreds of scripts for it.

Used here: http://sourceforge.net/projects/mystix/

Lazarus port: https://bitbucket.org/etrusco/lazarus-ide/src/22ba3d2ea4b7/components/synunihighlighter

Latest version: http://rghost.ru/46404542
Scripts: http://rghost.net/46404543