[SOLVED][SynEdit] GetNextHighlighterTokenFormView: Token starts with char

[Pascal]

How can i prevent this error when typing multibyte characters?
I use a SynEdit with a modified Cobol-Highlighter

Pascal

[jc99]

First read  http://wiki.lazarus.freepascal.org/Lazarus_Faq#What_is_the_correct_way_to_ask_questions_in_the_forum.3F
Meaning: What version, what OS, Some Code ...
Maybe a Test-Project, ...
So we can verify your bug, maybe you found a bug in the component, or it's just not released for those chars.     

[Pascal]

Sorry!

Version is 1.4.0 on Windows 8.1

[Basile B.]

Next time don't forget that SynEdit has its own subforum: http://forum.lazarus.freepascal.org/index.php/board,69.0.html

[Martin_fr]

I use a SynEdit with a modified Cobol-Highlighter

Does it happen with the not modified HL too?

In any case the error message is not worded very helpful, what it means is, that the highlighter returned a pointer to a token (part of the line text).
But the pointer points into the middle byte of utf8 codepoint.

In other words the HL tries to split a single char, so that the first byte(s) of it have one color, and the remaining byte(s) another.

[Pascal]

@BBasile: Thanks for the hint. I didn't recognized it so far.

@Martin_fr: Thanks for the explenation. That makes sense as the Cobol-Highlighter doesn't work with multi-byte-characters.
It steps byte wise thru the line which works if you do not use äöüÄÖÜß....
Is there a way to step column wise thru the line (line[collumn]) or do i have to use a char-pointer to handle multi-byte-characters?

Regards
Pascal

[Pascal]

I found the solution in TSynPasSyn Pascal Highlighter:

Just added:

Code: [Select]

   while (fLine[Run] in [#128..#191]) OR // continued utf8 subcode
   ((fLine[Run]<>#0) and (fProcTable[fLine[Run]] = @UnknownProc)) do inc(Run);
to my UnknownProc.

Thanks for the hints.

Regards
Pascal