Recent

Author Topic: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}  (Read 111544 times)

Martin_fr

  • Administrator
  • Hero Member
  • *
  • Posts: 5716
    • wiki
from sourcesynEditor
Code: Pascal  [Select]
  1.       List := TextView.FoldProvider.NestedFoldsList;
  2.       List.ResetFilter;
  3.       List.Clear;
  4.       List.Line := CaretY-1;
  5.       List.FoldGroup := FOLDGROUP_PASCAL;
  6.       List.FoldFlags := [sfbIncludeDisabled];
  7.       List.IncludeOpeningOnLine := False;
  8.  
  9.       InfCnt := List.Count;
  10.       for i := InfCnt-1 downto 0 do begin
  11.         NodeFoldType := TPascalCodeFoldBlockType({%H-}PtrUInt(List.NodeFoldType[i]));
  12.         if not(NodeFoldType in
  13.            [cfbtClass, cfbtClassSection, cfbtProcedure])
  14.         then
  15.  

this goes through the list and look for procedures and class and public/private
You can do the same, but check for begin, while, repeat, ....

Then check the info you can get from the object. Code navigation should show it.

Note the list returns pointers because it could be used for none pascal HL too, therefore the typcasts.

--------
List.Line
Quote
property Line: TLineIdx read FLine write SetLine;
"Idx" that means almost always 0 based.

While a name ending in ...Pos should be 1 based.

there are helpers in SynEditMiscProcs
function ToIdx(APos: Integer): Integer; inline;
function ToPos(AIdx: Integer): Integer; inline;

if you need to go from 0 to 1 based or vice versa, then use them. That is much clearer than  Y+1 or Y-1
They are new, so lots of code does not yet use them
« Last Edit: October 25, 2015, 02:07:17 am by Martin_fr »

x2nie

  • Sr. Member
  • ****
  • Posts: 478
  • Impossible=I don't know the way
    • impossible is nothing - www.x2nie.com
Personally, I'm no sure if many colors on the editor, can help on visualizing better the code. I prefer to have background colors.

Vertical lines, can be useful. I see it like an extension of the folding marks. ...

there is really not many colors ... a color for a certain block depth & most of the time there is maximum 4 colors  ...


Personally, I'am sure many color helped me on understanding pascal codes better.
Specially when I work in too many deep of nested begin..end (see the attachment for sample)


Hey, I am counting and there are 6 colors used. so, while there 6 colors (last color) exceeded , that 6 colors used again from beginning.
When you were logged in, you can see attachments.
Lazarus Trunk @ Windows7 64bit, XP 32bit, Debian under VirtualMachine

x2nie

  • Sr. Member
  • ****
  • Posts: 478
  • Impossible=I don't know the way
    • impossible is nothing - www.x2nie.com
Anyway, still about this colouring topic, current Lazarus is better (smart) than newest version of CnPack
when deal with {$IFDEF}.
It's important aspect, because ignoring this part, the whole work of nested-block-coloring will just guides programmer into wrong logic/algorithm. (see the screenshot for detail)


I also attached 2 file that can be used for later test.(not allowed by this forum's robot)
The gr32.pas and gr32_resamplers.pas is part of graphics32 library. it can be downloaded from such from https://searchcode.com/codesearch/view/62389395/






Anyway, if somebody could get any progress, can I collaborate? perhaps you must create another repository in sourceforge/svn with simple application +1SynEdit ?  8-)
When you were logged in, you can see attachments.
Lazarus Trunk @ Windows7 64bit, XP 32bit, Debian under VirtualMachine

x2nie

  • Sr. Member
  • ****
  • Posts: 478
  • Impossible=I don't know the way
    • impossible is nothing - www.x2nie.com
..
There is a method PrepareMarkupForRow in which you should prepare this list.
--------------------

There will be calls to
    Procedure GetNextMarkupColAfterRowCol(const aRow: Integer;
                                          const aStartCol: TLazSynDisplayTokenBound;
                                          const AnRtlInfo: TLazSynDisplayRtlInfo;
                                          out   ANextPhys, ANextLog: Integer); virtual; abstract;


Find the first pos after aStartCol on which you need to draw a line, in your list of blocks  and return it.

and in GetMarkupAttributeAtRowCol
wait until that column is due. then return a markup with a frame on the left side only (and set the frameStartBound (search examples in other markup)


Hi Mr. Martin, I did exactly what you said. It done nicely. (see attachment)
And yes, I agree that it should be in TSynEditMarkup, so it can also be used later in any TSynHighliter; rather than dedicated pascal highlighter only.


Now, the problem is it only care about the folding signal.
So, how we can continue this progress to care of pascal keywords (while, do, if, then .. ) ?


------
Anyway, I have different approach:
I currently don't care about begin-end pairs, because I don't want to draw the vertical lines yet. (Sure I will do that later.)
Instead, I only care about the level of any "folding" found in the line. <--- this level info is already calculated, so I don't do any calculation.
and I simply give a color related to the level:


Code: Pascal  [Select]
  1.     if sfaOpen in TmpNode.FoldAction then
  2.       lvl := TmpNode.FoldLvlStart
  3.     else
  4.       lvl := TmpNode.FoldLvlEnd;
  5.     ColorIndex := lvl mod (length(Colors));    
When you were logged in, you can see attachments.
Lazarus Trunk @ Windows7 64bit, XP 32bit, Debian under VirtualMachine

Martin_fr

  • Administrator
  • Hero Member
  • *
  • Posts: 5716
    • wiki
Cool, I will try to find some time to review it in more detail.

One thing I noted: You only look for the first TmpNode in "TmpNode := NodeList[ i ];". But there can be more than one "begin" in a line. Is that intended?
You could search for the next one in GetNextMarkupColAfterRowCol, once you are past the first. You would need to keep the list, release it in EndMarkup (and in PrepareRow, if none nil, or if in PrepareRow you got it already, just change the line.).
This would also fix the not marked "end" in your screengrab.

Quote
Now, the problem is it only care about the folding signal.
So, how we can continue this progress to care of pascal keywords (while, do, if, then .. ) ?

if/then/else/; is actually available (depends on filter settings), It is not so much for folding, but the HL must keep track of nested if, in order to differentiate between a case-else and if-else.

It is disabled. In the IDE it can be turned on via the ide options, but in SynEdit you need to add some code, I need to search myself. try "NodeList.FoldFlags:= [sfbIncludeDisabled]" instead

-----------
"while" should ideally not be added as fold-level.  Adding more items to the fold-level increases the resources the HL will need.

Though if it is the ONLY one missing it may have do be discussed.

How would/should a "while" look? without begin/end it has no end-token (well the ";" terminates it)?
Of course once you have vertical lines, there is a use.

Is "while" the only one missing?

How much nesting support does it need? All while (if without begin/end) end in the same place, at the ";"
Code: Pascal  [Select]
  1.   while a do
  2.     while b do
  3.       foo();
-----------

if an "if" or "while" has a begin/end: will they both highlight?

If you have a begin block, you can check the outer block.

But maybe better, the HL has sfaMarkup to indicate word pair highlight. It may introduce sfaMarkupOutline. Then the HL can determine which nodes to outline and which not. (so the markup will stay independent)

For info, each nodes has flags
Code: Pascal  [Select]
  1.   TSynFoldAction = (
  2.                      sfaOpen,         // Any Opening node
  3.                      sfaClose,        // Any Closing node
  4.  
  5.                      sfaFold,         // Part of a fold- or hide-able block (FoldConf.Enabled = True)           - excludes one=liners for FoldFold, as they can not fold
  6.                      sfaFoldFold,     // Part of a fold-able block (FoldConf.Enabled = True / smFold in Modes)  - excludes one=liners / only opening node, except ifdef/region (todo: maybe both?)
  7.                      sfaFoldHide,     // Part of a hide-able block (FoldConf.Enabled = True / smHide in Modes)  - includes one=liners / only opening node, except ifdef/region (todo: maybe both?)
  8.  
  9.                      sfaMultiLine,    // The closing node is on an other line
  10.                      sfaSingleLine,   // The closing node is on the same line (though the keyword may be on the next)
  11.                      // //sfaSingleLineClosedByNext
  12.                      sfaCloseForNextLine,  // Fold closes this line, but keyword is on the next (e.g. "var" block)
  13.                      sfaLastLineClose,     // Fold is incomplete, and closed at last line of file
  14.  
  15.                      sfaDefaultCollapsed,
  16.                      sfaMarkup,   // This node can be highlighted, by the matching Word-Pair Markup
  17.                      sfaInvalid,  // Wrong Index
  18.  

x2nie

  • Sr. Member
  • ****
  • Posts: 478
  • Impossible=I don't know the way
    • impossible is nothing - www.x2nie.com
Hi Mr. Martin


* the single line 'begin~end' has been resolved.
* The [sfbIncludeDisabled] doesn't work, because No method ables to make it applicable to NodeList.
* Yes, nested "while-do--while-do;" can be painted by one color. Therefore "if-then-else--if-then-else" too;
The reason is because both is just one pascal statement ( = closed with with one " ; ")
But, nested routine isn't. Meaning...
Code: Pascal  [Select]
  1. procedure funcParent();
  2.    procedure funcChild();
  3.    begin
  4.       ...
  5.    end;
  6. begin
  7. ...
  8. end;
The begin+end part of a routine should be painted in differently color compared to begin+end part of it's children.
It's needed for notice us that the begin+end block of child is not yet the main block of the parent function.


* I got a few more progress, I can now coloring the several pascal keyword without any call to specific (pascal) highlighter.


----
I also found another problem: the higlighter reports inconsistency in if~then block. see the attachment.
I don't know why the 'if' is painted in similar level with 'begin', and in another line it is painted differently.


----
I am now working in drawing vertical lines.
I found that most efficient way to draw these lines is by: just reading the cache.
so I will move the PrepareMarkupForRow's content into DoTextChanged.
Further I plan the cache should be progresif: continue find the 'end' part of folding sytem only if needed.
such when user scroll down the page. in other word: we don't need to build the cache of the whole lines, only the was visible area will be calculated.


----
I also play with the TSynFoldAction. The cons is it work only with 'and' mode, while I need the 'or' mode.
So, NodeList.ActionFilter := [sfaSingleLine,sfaMultiLine];  (or similar combination)  doesn't work.
THe only way I can reach is by call filtering twice. It's not a problem for now, I just dream that there are a simpler way. 8-)


The funny thing: It's running well: NodeList.ActionFilter := [];  8-)
So, I keep this way.


-----
The cons of current TSynPascalSyn is : It doesn't report the "with~do" as fold signal, while the "repeat~until" works.

When you were logged in, you can see attachments.
Lazarus Trunk @ Windows7 64bit, XP 32bit, Debian under VirtualMachine

x2nie

  • Sr. Member
  • ****
  • Posts: 478
  • Impossible=I don't know the way
    • impossible is nothing - www.x2nie.com
if an "if" or "while" has a begin/end: will they both highlight?
Yes it is. Maybe they (if-then-begin-end-else-begin-end) should be painted in one color per level.
But, configurable (highlighted or not) may be better for other user.

If you have a begin block, you can check the outer block.

But maybe better, the HL has sfaMarkup to indicate word pair highlight. It may introduce sfaMarkupOutline. Then the HL can determine which nodes to outline and which not. (so the markup will stay independent)
sfaMarkupOutline is not found in current lazarus trunk, okay?
I think implement it will solve the problem (my previous post).
It will also make the synedit folding system more flexible and clean.

---
Sorry for was not understand some parts of your post. Therefore I report problem that already has given clues by you in several post.
I only understood them after trying to code my self. >:D
And while reading again your post, they (what you said) seem clearer for me.  ;) :D
« Last Edit: December 02, 2015, 01:53:05 pm by x2nie »
When you were logged in, you can see attachments.
Lazarus Trunk @ Windows7 64bit, XP 32bit, Debian under VirtualMachine

x2nie

  • Sr. Member
  • ****
  • Posts: 478
  • Impossible=I don't know the way
    • impossible is nothing - www.x2nie.com

--------------

When this works there may be a need to add things to the painter, because if a vertical line is dotted or dashed, it will not look good. A new fragment of it is painted for each text line, and that resets the dotted pattern.

This needs to be fixed in the painter, but this is for the very end.

At this moment the only thing that goes outside is a fix in the painter for dotted vertical lines continued over several lines of text.
They still need to be drawn in individual bits per text line, in order to obey priority against other lines. The painter can use clipping to ensure they are joined correctly
I disagree.
I have different approach that doesn't require the change of painter:
* We can use "GetMarkupAttributeAtRowCol()" as a command instruction to painter, and let the only painter do painting work.
* what we send to painter is just as usually (X,X2, colorForeground, backgroundColor, framecolor).
The trick is we send instruction to the painter to paint only the left border on given highlighted area.


* but the only problem is: how to send the "bsLeft" part of  TLazSynBorderSide ? Is the TSynEditMarkup.MergeMarkupAttributeAtRowCol the right place to do that?  :o


« Last Edit: December 02, 2015, 02:32:09 pm by x2nie »
When you were logged in, you can see attachments.
Lazarus Trunk @ Windows7 64bit, XP 32bit, Debian under VirtualMachine

Martin_fr

  • Administrator
  • Hero Member
  • *
  • Posts: 5716
    • wiki
Quote
* The [sfbIncludeDisabled] doesn't work, because No method ables to make it applicable to NodeList.
property FoldFlags: TSynFoldBlockFilterFlags

Quote
* Yes, nested "while-do--while-do;" can be painted by one color. Therefore "if-then-else--if-then-else" too;
Quote
the higlighter reports inconsistency in if~then block. see the attachment.
if-then-else are 2 or even 3 blocks. The "else" is in both.
- if: open
- then: close / open another
- else: close / open another
- ; end or other closing reason: close

So each of them has 2 entries in  the node-list.

Quote
I am now working in drawing vertical lines.
I found that most efficient way to draw these lines is by: just reading the cache.
What cache?
Quote
so I will move the PrepareMarkupForRow's content into DoTextChanged.
DoTextChanged can be called a lot more often. THat is not a good idea. It can also be called for of screen changes.
It is meant to be used to invalidate cached data, usually be setting a flag (which does not cost time) "InvalidateNeeded := true;"

Quote
I also play with the TSynFoldAction. The cons is it work only with 'and' mode, while I need the 'or' mode.
So, NodeList.ActionFilter := [sfaSingleLine,sfaMultiLine];  (or similar combination)  doesn't work.
Then do not filter at all, and filter the result you read. Or suggest a way to add it to the list, and a patch can be discussed.
Quote
The funny thing: It's running well: NodeList.ActionFilter := [];  8-)
So, I keep this way.
Yes, IIRC filter only filters, if at least one filter is set.

Quote
The cons of current TSynPascalSyn is : It doesn't report the "with~do" as fold signal, while the "repeat~until" works.
ok, so "with" and "while" are both missing.

I currently am a bit time pressed, so I have to answer that later.


Quote
Quote from: Martin_fr on December 01, 2015, 11:44:22 pm

    if an "if" or "while" has a begin/end: will they both highlight?

Yes it is. Maybe they (if-then-begin-end-else-begin-end) should be painted in one color per level.
But, configurable (highlighted or not) may be better for other user.
the HL can be modified to deliver correct info for that, later more.

that is what sfaMarkupOutline is about. It needs to be ADDED. then the HL can set it as needed. (inclusive user conf)

more later


Martin_fr

  • Administrator
  • Hero Member
  • *
  • Posts: 5716
    • wiki
When this works there may be a need to add things to the painter, because if a vertical line is dotted or dashed, it will not look good. A new fragment of it is painted for each text line, and that resets the dotted pattern.

This needs to be fixed in the painter, but this is for the very end.

At this moment the only thing that goes outside is a fix in the painter for dotted vertical lines continued over several lines of text.
They still need to be drawn in individual bits per text line, in order to obey priority against other lines. The painter can use clipping to ensure they are joined correctly
I disagree.
I have different approach that doesn't require the change of painter:
* We can use "GetMarkupAttributeAtRowCol()" as a command instruction to painter, and let the only painter do painting work.
* what we send to painter is just as usually (X,X2, colorForeground, backgroundColor, framecolor).
The trick is we send instruction to the painter to paint only the left border on given highlighted area.
Yes you use the frame left site.

What I meant is, if you use a dotted line, the painter will sometimes join it incorrectly causing tiny interruptions. But that can be fixed later.


Quote
* but the only problem is: how to send the "bsLeft" part of  TLazSynBorderSide ? Is the TSynEditMarkup.MergeMarkupAttributeAtRowCol the right place to do that?  :o

You sent it from GetMarkupColAtRowCol, like the color. Set the frame color and frame side, and check code in the other markup. you need to set
MarkupInfo.SetFrameBoundsLog(x1, x1)


x2nie

  • Sr. Member
  • ****
  • Posts: 478
  • Impossible=I don't know the way
    • impossible is nothing - www.x2nie.com
Quote
I am now working in drawing vertical lines.
I found that most efficient way to draw these lines is by: just reading the cache.
What cache?
Quote
so I will move the PrepareMarkupForRow's content into DoTextChanged.
DoTextChanged can be called a lot more often. THat is not a good idea. It can also be called for of screen changes.
It is meant to be used to invalidate cached data, usually be setting a flag (which does not cost time) "InvalidateNeeded := true;"


Well, that means we still use PrepareMarkupForRow?, and DoTextChanged  was not the right place to save the open~close--open~close folding list through several lines?
IMHO, Then the problem (with current version of SynEdit) is: identifiying of verticals line couldn't be efficient.

You know, in TSynEditMarkupWordGroup we search at least one pair of begin~end.
But in my TSynEditMarkupFoldColors, below sample require at least 5 pair begin~end for getting the correct position of it's 5 vertical lines:

| |    | | | HorzEntry := EMPTY_ENTRY;
(see attachment for complete lines)

Sure, we can do searching these 5 pairs on PrepareMarkupForRow, but AFAIK its not efficient in a long procedure/code.
In other words, why we should repeat finding all pairs per  line, while we can do that once per screen changes (DoTextChanged) ?
I choose DoTextchanged because it is the only chance before iteration of each line.
DoTextChanged is also paired with MarkupFinish(); Maybe I need something new like MarkupStart() that works outside the iteration of painting each lines.
Simply: When DoTextChangedMarkupStart is not the right place, IMHO PrepareMarkupForRow is even worst.


I think any other TSynMarkup is useless for comparing with TSynEditMarkupFoldColors's job
Thats why I planned that special case.


-----
But, wait...
There is another possible choice, which may best to do:
If PrepareMarkupForRow is really the only chance to identify the position of 5 vertical lines,
so it can be very efficient if there was no calculation/searching the pairs at all
in PrepareMarkupForRow.



How it will be done?
The highlighter (TSynCustomFoldHighlighter) should able to provide those 5 vertical lines's position !  8-) >:D
This way, the job done in PrepareMarkupForRow by simple TempNode := NodeList[ i ];
The real calculation is in TSynCustomFoldHighlighter (or descendant) which is the correct class who know how to.


Further, for simplify the TSynCustomFoldHighlighter's filter when deal without vertical-lines's requirement,
 I guess it should be something config, like : FoldFlag := [sbfIncludeDisable, sbfIncludeParents] ?


Quote
What cache?
The NodeList thats done calculated outside the PrepareMarkupForRow, that is cache in my mind.
« Last Edit: December 03, 2015, 01:36:55 am by x2nie »
When you were logged in, you can see attachments.
Lazarus Trunk @ Windows7 64bit, XP 32bit, Debian under VirtualMachine

Martin_fr

  • Administrator
  • Hero Member
  • *
  • Posts: 5716
    • wiki
Quote
Well, that means we still use PrepareMarkupForRow?, and DoTextChanged  was not the right place to save the open~close--open~close folding list through several lines?
Yes, even so that I have not yet seen what is in DoTextChanged.


Quote
IMHO, Then the problem (with current version of SynEdit) is: identifiying of verticals line couldn't be efficient.

It should. Look at the word-pair (tripplet) markup.

Search of a matching fold node is very efficient. You can find the correct line without looking into the line, simply by looking at the level.
If you search ends for several begin, you know that the next end is further down, so you can continue in the found line.

You could cache this info (and invalidate in DoTextChanged). But you may need a real big cache. there could be several 100 visible lines. (small font, big screen).
Or there could be a lot of folded lines, so the last line visible in the window is 5000 lines below the first (4950 lines folded).

You definitely should cache between lines, during one screen refresh/paint. That is if you calculated for line 390, then you cache because likely 391 is next (unless folded). Cache is then cleared in EndMarkup.

Look at TLazSynEditNestedFoldsList. This list provide ALL open fold nodes at the start of a line
Code: Pascal  [Select]
  1. procedore foo,
  2. var
  3.   i: integer;
  4. begin
  5.   for a := 1 to 2 do begin
  6. // TLazSynEditNestedFoldsList will provide both begin, and the procedure, but not the var
  7.  
  8.  

The searching of matching end nodes could be added (not yet there) to this list. It is useful in many places.

When the first PrepareRowFor... is called, you get all open blocks (open from previous lines) from the list.
During the row you modify as required.

If rows are skipped (folded), then you ask TLazSynEditNestedFoldsList again. (you can merge with what you know from the above rows)

from markup
Code: Pascal  [Select]
  1. function TSynMarkupHighIfDefLinesTree.GetHighLighterWithLines: TSynCustomFoldHighlighter;
  2. begin
  3.   Result := FHighlighter;
  4.   if (Result = nil) then
  5.     exit;
  6.   Result.CurrentLines := FLines;
  7. end;
  8.  
  9. function TSynMarkupHighIfDefLinesTree.CreateOpeningList: TLazSynEditNestedFoldsList;
  10. begin
  11.   Result := TLazSynEditNestedFoldsList.Create(@GetHighLighterWithLines);
  12.   Result.ResetFilter;
  13.   Result.Clear;
  14.   //Result.Line :=
  15.   Result.FoldGroup := FOLDGROUP_PASCAL = ;
  16.   Result.FoldFlags := [sfbIncludeDisabled];
  17.   Result.IncludeOpeningOnLine := False;
  18. end;
  19.  

also used in ide/sourcesynEditor.pas
TIDESynEditor.SrcSynCaretChanged


Martin_fr

  • Administrator
  • Hero Member
  • *
  • Posts: 5716
    • wiki
Sorry, before my last post I mixed it up, and thought the list you used and TLazSynEditNestedFoldsList  where the same.
hence my post about sfbIncludeDisabled

x2nie

  • Sr. Member
  • ****
  • Posts: 478
  • Impossible=I don't know the way
    • impossible is nothing - www.x2nie.com
Yes! you right.
everything can be done in PrepareMarkupForRow();


Now, the problem goes to TSynPascalSyn:
* The begin~end level of "if" is seem wrong.
* But, the begin~end level of "else" is correct.


Any idea ?


Another problem:
* I can not set left-only-border. The 4 side border always drawn.
   its because GetMarkupAttributeAtRowCol ( you told as GetMarkupColAtRowCol) is only work with TSynSelectedColor, while we need TSynSelectedColorMergeResult.
I have no clue how to instruct painter to draw only the left side.


Screenshots are only deal TLazSynEditNestedFoldsList.
I temporary disable the TLazSynFoldNodeInfoList (NodeList). But they are yet beautiful enough for now.

------
The code can be downloaded:
web: https://github.com/x2nie/syneditmarkupnestedcolors
svn: https://github.com/x2nie/syneditmarkupnestedcolors
git: https://github.com/x2nie/syneditmarkupnestedcolors
zip: https://github.com/x2nie/syneditmarkupnestedcolors/archive/master.zip

« Last Edit: December 03, 2015, 06:28:08 am by x2nie »
When you were logged in, you can see attachments.
Lazarus Trunk @ Windows7 64bit, XP 32bit, Debian under VirtualMachine

x2nie

  • Sr. Member
  • ****
  • Posts: 478
  • Impossible=I don't know the way
    • impossible is nothing - www.x2nie.com
Further more progress / bugfix:
if-begin-end vs else-begin-end coloring bug was my fault !! :-[  I've resolved that.


Now, the problem of nested-color is the ChildProcedure-begin-end. It should has different level than the MainProcedure-begin-end.
When you were logged in, you can see attachments.
Lazarus Trunk @ Windows7 64bit, XP 32bit, Debian under VirtualMachine