Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}

[A-M]

Hi Dear FPC/Lazarus Developers
The idea is so simple , colorizing code blocks start and ending keywords for helping on perception while developing and browsing the source code
like what the CNPack add-on do on delphi IDE

its specially important when you are developing a complex algorithm with multi level code blocks inside ...
and even i can say it will be effective for choosing lazarus/fpc over other languages and IDEs for programming newcomers.



i'm ready to start the job ,
if you know what the class or method is suitable for adding this feature
or you want help doing this 
or even you have any comment about , i will be happy to know.


Best Regards.

[Martin_fr]

Hi

I am the current maintainer of SynEdit in Lazarus.

   Welcome.

Nice to have someone interested in helping.


This should be done as a new subclass of TSynEditMarkup.

Colors can be added to TSynSelectedColor. look at frames and frame boundaries. Then the drawer will deal with them. (It needs testing for drawing in the middle of chars, which can happen if the line goes through a tab)

Look at SynEditMarkupWordGroup to see how to find Begin and end of blocks efficiently.


I will gladly add more help later.

-----------------
Things to start with

You need to understand:

http://wiki.lazarus.freepascal.org/SynEdit_Highlighter
How "ranges" are used in folding, and how they "level" can help finding blocks

Basics about what tokens are (this is how the HL splits the line). You probably will not need details.

http://wiki.lazarus.freepascal.org/SynEdit
logical and physical caret. this is essential.

Have a look at TSynSelectedColor and what it already provides

[Martin_fr]

this is also good http://wiki.lazarus.freepascal.org/Redesign_of_the_SynEdit_component

Not 100% up to date, and not 100% related, but an overview how things connect.

[Martin_fr]

SynEditMarkupWordGroup  really is the start point. It is the code that highlights matching begin / end, if the cursor is on it

When you write your code, you will need to find all begin/end (try finally / repeat until / ...) pairs (iirc you can even find if/then this way).

Of course in

Code: Pascal  [Select][+][-]

1. while a
2. do
3. begin
4.   //...
5. end;
6.  

you then get a line for begin end, not starting at the while (same as code folding does).

So the first task is to find all those blocks in the current display. And it must be efficient, with folding there can be 1000 lines between top and bottom line.
For that you use the HL level info. (and later tricks like caching, but not to worry now)

When you have all the blocks surrounding the current line, then you know where to draw the lines. (Also when you have all blocks for line N, remember them, and if the next line is N+1 you can check which ones to re-use.)

There is a method PrepareMarkupForRow in which you should prepare this list.
--------------------

There will be calls to
    Procedure GetNextMarkupColAfterRowCol(const aRow: Integer;
                                          const aStartCol: TLazSynDisplayTokenBound;
                                          const AnRtlInfo: TLazSynDisplayRtlInfo;
                                          out   ANextPhys, ANextLog: Integer); virtual; abstract;


Find the first pos after aStartCol on which you need to draw a line, in your list of blocks  and return it.

and in GetMarkupAttributeAtRowCol
wait until that column is due. then return a markup with a frame on the left side only (and set the frameStartBound (search examples in other markup)

[AlexTP]

Martin Fr.

Look at your msg.

You write each sentence, almost each, on new line.

Even not new line.
Even on new paragraf.

This is irritating for usual reader. IMHO.

[kapibara]

On the contrary I think, it's very readable.

[Martin_fr]

reduced a few of them. But basically it is a bullet point list, without the markup

[A-M]

Thank you guys for your attention
and specially thank martin for welcoming me
i'm reading the documentation and codes and will post a summary of my survey soon .

[A-M]

Summary [updating]:

1-the whole job consist of two parts :
1-1-coloring the keywords with different colors on different depths that seems to be possible just by extending the Syntax Highlighter class
1-2-drawing the vertical lines behind the code blocks that seems to need editing the main TSynEdit class


2- TSynCustomFoldHighlighter is most complete highlighter class that consist Folding & Highlighting information together , TSynPasSyn that logically should be the Lazarus IDE highlighter extended from this class , this class has a property named "KeyAttri"(KeywordAttribute) of TSynHighlighterAttributes class that defined on SynEditHighlighter Unit;
and finally TSynHighlighterAttributes consist of some attribute including foreground color , background color and etc ...
As we need to coloring the Keywords on each depth with different colors we need "KeyAttri" to be an Array of  TSynHighlighterAttributes

[molly]

pardon the intrusion, you meant something similar like SynUniHighlighter or SynFacilSyn ? (first seems hard to find, here perhaps ?)

[Martin_fr]

Unfortunately it is not the Highlighters job.

The highlighter is doing the basic parsing. Though the line is not 100% clear in this case. But the Markup is match better suited.

The Paint class paints the text token by token. It pulls them from the HL but they have a long way.

unit LazSytTextArea

Code: Pascal  [Select][+][-]

1. function TLazSynPaintTokenBreaker.GetNextHighlighterTokenFromView(out
2.   ATokenInfo: TLazSynDisplayTokenInfoEx; APhysEnd: Integer; ALogEnd: Integer): Boolean;
3. ...
4.   function MaybeFetchToken: Boolean; inline;
5.   begin
6. ....
7.     while FCurViewToken.TokenLength = 0 do begin // Todo: is SyncroEd-test a zero size token is returned
8.       Result := FDisplayView.GetNextHighlighterToken(FCurViewToken);
9.  

FDisplayView is a TLazSynDisplayView

It comes from the Highlighter and passes through the TextBuffer and various Views (such as FoldedView). On each step it can be modified, or even tokens be added.

unit LazSytTextArea

Code: Pascal  [Select][+][-]

1. function TLazSynPaintTokenBreaker.GetNextHighlighterTokenEx(out
2.   ATokenInfo: TLazSynDisplayTokenInfoEx): Boolean;
3. ...
4.     FMarkupManager.GetNextMarkupColAfterRowCol(FCurTxtLineIdx+1,
5.  

Calls the markup manager, to apply final markup.
This includes selection, visible whitespace-color, current word markup, ....

Now nested begin end can be seen as pascal grammar specific, but the kind of markup can work with any HL that has folding. It could work with LFM too.

So it really should be in a Markup.

Also you have markup on lines that between the keywords. The HL has no business to calculate them, and the painter should neither. This calculation goes into the Markup class.

--------------

The markup can and should find all blocks surrounding the line currently painted.
It can then:
- modify tokens for begin/end (as SynEditMarkupWordGroup  does)
- modify tokens on the line between, that need a vertical line.

--------------

When this works there may be a need to add things to the painter, because if a vertical line is dotted or dashed, it will not look good. A new fragment of it is painted for each text line, and that resets the dotted pattern.

This needs to be fixed in the painter, but this is for the very end.

[Martin_fr]

For reference. This are the current markup classes:
  SynEditMarkup, SynEditMarkupHighAll, SynEditMarkupBracket, SynEditMarkupWordGroup,
  SynEditMarkupCtrlMouseLink, SynEditMarkupSpecialLine, SynEditMarkupSelection,
  SynEditMarkupSpecialChar,

And 99% of your code will be in a new sub class of TSynEditMarkup.

At this moment the only thing that goes outside is a fix in the painter for dotted vertical lines continued over several lines of text.
They still need to be drawn in individual bits per text line, in order to obey priority against other lines. The painter can use clipping to ensure they are joined correctly
I would not worry about that now and concentrate on the Markup only

[Martin_fr]

There also is TLazSynEditNestedFoldsList which can help finding begin/end and other fold nodes in the HL.

It is used in ide/SourceSynEditor

[A-M]

Unfortunately it is not the Highlighters job.

The highlighter is doing the basic parsing. Though the line is not 100% clear in this case. But the Markup is match better suited.

The Paint class paints the text token by token. It pulls them from the HL but they have a long way.

i'm a bit confused from your post,may be because i have no background on TSynEdit
but i strongly believe at least half of work should be in HL
the below code as you know is GetTokenAttribute function in Pascal HL

Code: Pascal  [Select][+][-]

1. function TSynPasSyn.GetTokenAttribute: TSynHighlighterAttributes;
2. begin
3.   case GetTokenID of
4.     tkAsm: Result := fAsmAttri;
5.     tkComment: Result := fCommentAttri;
6.     tkIDEDirective: begin
7.       FCurIDEDirectiveAttri.Assign(FCommentAttri);
8.       FCurIDEDirectiveAttri.Merge(FIDEDirectiveAttri);
9.       Result := FCurIDEDirectiveAttri;
10.     end;
11.     tkIdentifier: Result := fIdentifierAttri;
12.     tkKey: Result := fKeyAttri;
13.     tkNumber: Result := fNumberAttri;    
14.     ...
15.  

i think the work is easy as doing these steps on Pascal HL :
1-change fKeyAttri definition from TSynHighlighterAttributes to Array of TSynHighlighterAttributes .
2-Defining a variable for keeping current token depth that will increase on StartCodeFoldBlock and decrease on EndCodeFoldBlock.
3-finally doing the change on GetTokenAttribute like this :
tkKey: Result := fKeyAttri[CurrentTokenDepth];

why do you think this is not correct ?
i agree that there is a long way from HL to paint class but what is the problem ?

However i have no idea about drawing that vertical lines till now ...

[A-M]

pardon the intrusion, you meant something similar like SynUniHighlighter or SynFacilSyn ? (first seems hard to find, here perhaps ?)

you are welcomed to take a part in discussion
please look at the first post attachment to see what we want
i didn't found same thing on your links.