tsynanysyn not highlighting properly after upgrade

[Michael Collier]

I recently upgraded to lazarus 1.6 / fcl3.0.0 and now my tsynanysyn doesn't work properly.

My list of keywords that should show up bold now only shows up some of the keywords.

Here is my list of keywords, if you paste them into tsynanysyn keyword list, and then paste them into a synedit to view, you will see patches where some words are not highlighted.

I checked the lfm file and they are all stored uppercase without spaces, can't see why they suddenly don't all work..

edit: Attached example project in a lower thread..

[Edson]

I don't know about TSynAnySyn, but maybe you could use my Highlighter:  https://github.com/t-edson/SynFacilSyn

I haven't tested in Lazarus 1.6 but someone told me, it works, with a little changes.

[Martin_fr]

You need to provide a full example (project1, unit1), with minimum code to reproduce. (including the text in SynEdit in which to HL_

Drop a SynEdit on a form, add the HL, add text and keywords, check the error is present, zip it, attach it.

[Michael Collier]

Attached screenshot and simple project, thanks.

[Michael Collier]

I just tried re-opening my attached project but lazarus reported that the lpi file didn't look right, so I opened broken_syn.lpi and found it to be completely empty. Seems lazarus IDE didn't save it properly, so I copied broken_syn.lpi.bak into it and can use the project. Not sure why it did that, I didn't do anything unusual, just created/ran/saved project.

[Martin_fr]

Please report a bug.  (and refer to this post)

Unfortunately I dont have enough time to fix it right now.

function TSynAnySyn.IsKeyword(const AKeyword: string): boolean;             

is wrong.
Probably AnsiCompareStr uses a different sort order that TStringList (the latter may have changed).

So this seach needs to be rewritten, or better TStringLists sorting needs to be done by the same order (overwrite TStringLists.DoCompare or similar)

----------------
A quick workaround, is to change this method to let stringlist do the search.

[guest58172]

By the way it could be faster to store the KWs in a hash map.

[vfclists]

By the way it could be faster to store the KWs in a hash map.

There is an interesting suggestion. Hash maps I something I read about about often but I don't know what they are. What I understsnd is they are meant to speed up string lookups. Do you have or know of any Pascal implentations of them?

[Michael Collier]

Thanks for the info, will report a bug, in the meantime I did a quick hack to the TSynAnySyn isKeyword function to see if it fixes my problem which it does:

Code: Pascal  [Select][+][-]

1. Result := ( fKeyWords.IndexOf ( AKeyword ) <> -1 ) ;  

I'll keep an eye on this thread for more elegant solutions..

Cheers.

[guest58172]

By the way it could be faster to store the KWs in a hash map.

There is an interesting suggestion. Hash maps I something I read about about often but I don't know what they are. What I understsnd is they are meant to speed up string lookups. Do you have or know of any Pascal implentations of them?

When you use a map as dictionary, there is always almost a single string comparison (e.g when there is only one value per bucket, which is the most common case.

To test the presence of a value, it's hashed and the dictionary looks if the hash is in the map. If not there's no string comparison at all, if yes the items in the bucket are compared.

But actually I've got a bad surprise while using the existing map in fgl:

Code: Pascal  [Select][+][-]

1. program Project1;
2. uses fgl, classes, sysutils;
3. type
4.   TDictionary = class(specialize TFPGMap<string, boolean>);
5. const
6.   kws: array[0..5] of string =
7.     ('absolute','relative','post','pre','increment','decrement');
8.   aws: array[0..5] of string =
9.     ('absorute','relative','prst','preert','inemnt','decrement');
10.   cnt = 10000000;
11.  
12. var
13.   dict1: TStringList;
14.   dict2: TDictionary;
15.   str: string;
16.   b: boolean;
17.   t: QWord;
18.   i: integer;
19.  
20. begin
21.  
22.   dict1 := TStringList.create;
23.   dict2 := TDictionary.create;
24.  
25.   for str in kws do dict2.Add(str);
26.   dict1.AddStrings(kws);
27.  
28.   write('TStringList as dictionary: ');
29.   t := GetTickCount64;
30.   for i := 0 to cnt do
31.     b := dict1.IndexOf(aws[i mod high(aws)]) <> -1;
32.   writeln(GetTickCount64 - t);
33.  
34.   write('TFPGMap as dictionary: ');
35.   t := GetTickCount64;
36.   for i := 0 to cnt do
37.     b := dict2.IndexOf(aws[i mod high(aws)]) <> -1;
38.   writeln(GetTickCount64 - t);
39.  
40.   dict1.Free;
41.   dict2.Free;
42. end.

The hashmap-based dictionary is twice slower !

[Martin_fr]

Look up TFPGMap in source. It maps each value to a key, and stores the keys in a sorted list.
So it still does the same binary search.

SynPluginSyncroEdit has a hash implementation: TSynPluginSyncroEditWordsHash

It has code only used for syncro edit, but a generic could be extracted from it.

[guest58172]

Fine, I'll try to propose a patch in the next weeks, it should be trivial to replace the string list.

[Edson]

SynFacilSyn is a very fast highlighter. Now you can use it with FPC 3.0.0 (I guess  ).