Recent

Author Topic: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}  (Read 111203 times)

Martin_fr

  • Administrator
  • Hero Member
  • *
  • Posts: 5691
    • wiki
Code: Pascal  [Select]
  1. TSynSelectedColor.FrameEdges := sfeLeft[
  2. /code]
  3. Should have been an enum, but isnt....
  4.  
  5. [quote]Now, the problem goes to TSynPascalSyn:
  6. * The begin~end level of "if" is seem wrong.
  7. * But, the begin~end level of "else" is correct.[/quote]
  8.  
  9. I had  a quick look. the issue is that there is one block from
  10. "if" to "then"
  11. and a 2nd block from
  12. "then" to "else"
  13.  
  14. or if there is no "else" it goes from "then" to ";"
  15.  
  16. But there is no block from "else" to ";". (that wasnt needed to get the "case a of 1: write ELSE read end;"
  17.  
  18. Not yet sure how to fix it best.
  19.  
  20. Also it currently the case there is actually no gurantee of monotonic increases.
  21. If the HL would need to keep track of something else, it might count up.
  22.  
  23. It can be added as extra counter, but that is probably more costly than counting it in the markup.
  24.  
  25. On the other hand we will need the else (like the "while") so maybe....
  26.  
  27. I have to come back on this. Might be a bit...
  28.  
  29. --------------
  30. But I do see you a making create progress :)

Martin_fr

  • Administrator
  • Hero Member
  • *
  • Posts: 5691
    • wiki
Further more progress / bugfix:
if-begin-end vs else-begin-end coloring bug was my fault !! :-[  I've resolved that.


Now, the problem of nested-color is the ChildProcedure-begin-end. It should has different level than the MainProcedure-begin-end.

It will be if you enable folding for "procedure"

Code: Pascal  [Select]
  1. SynEdit.FoldConfig[ord(cfbtProcedure)].Enabled := true

Or use NestLvlStart instead of FoldLvlStart, but that may increase too fast.

So you may need to keep your own count, when you go through the nodes, and decide which ones to use
« Last Edit: December 03, 2015, 06:48:26 am by Martin_fr »

x2nie

  • Sr. Member
  • ****
  • Posts: 478
  • Impossible=I don't know the way
    • impossible is nothing - www.x2nie.com
Hi Mr. Martin,


There is no different >> self.SynFreePascalSyn1.FoldConfig[ord(cfbtProcedure)].Enabled := True/False;


Anyway, I have good news:
the vertical lines finally drawn properly !  8-)
It is now dejavu. Very similar to CNPack.


But I found some bug made by me. Such as when a "begin" moved to the left (by deleting trailing space).
I will fix them later.


----
My priority (and curiosity) is now going to the highlighter it self.
I wonder, why SynHighlighterLFM doesnt reports any nested folding, sure the nested folding is enabled (I can click in the gutter) ?


I also  include the FoldHL.pas, it is the simplest highlighter I can found from the SynEditTutorial.
But it also doesn't reports any nested folding which can be used by SynEditMarkup.
What's I should add/modif to reach what SynHighligterPascal had has?  :o


----

The code can be downloaded:
svn: https://github.com/x2nie/syneditmarkupnestedcolors
git: https://github.com/x2nie/syneditmarkupnestedcolors
zip: https://github.com/x2nie/syneditmarkupnestedcolors/archive/master.zip

When you were logged in, you can see attachments.
Lazarus Trunk @ Windows7 64bit, XP 32bit, Debian under VirtualMachine

Martin_fr

  • Administrator
  • Hero Member
  • *
  • Posts: 5691
    • wiki
I have to get back on those questions.

Nice work, just downloaded.
-----------------------------
You need to add invalidation calls.

Add or remove space in front of a "begin" (then resize the editor to invalidate)


If markup changes for any line, then self.InvalidateSynLines(firstline, last);
must be called.

And that should happen before painting, which also means before GetNextMarkup......

So I was wrong on some of the previous info.

When markup.TextChanged is called you need to check if any line in the supplied range must be repainted, and send invalidate if so.

For testing you can invalide the entire screen, but for real only the minimum needed should be done. people run on laptops, and every action costs battery.


So to invalidate you need to know what you painted last, and what you paint next.
You need some structure that can store this, yet wont need to much memory.

maybe a TList of all lines.
  record
    x, y, color, heigth
  end

There may be other/better ways


Then in TextChanged you do what you currently do in GetMArkup..... and compare, invalidate, store the result.

Thinks outside the supplied line range have not changed, but if either begin or end are in the changed lines ....


--------------
More later




Martin_fr

  • Administrator
  • Hero Member
  • *
  • Posts: 5691
    • wiki
coming to think of it.
TextChanged means you have no indication of which lines are visible.

Imagine a file with 300.000 lines (yes such files exist) and 100.000 lines folded in the range you need to scan.

You can use SynEdit.LinesInWindow and then

Code: Pascal  [Select]
  1.   for i := 1 to SynEdit.LinesInWindow + 1 do begin
  2.     l := SynEdit.RowToScreenRow(i);
  3.     // do work for l, if l was in changed range
  4.  
  5.  

x2nie

  • Sr. Member
  • ****
  • Posts: 478
  • Impossible=I don't know the way
    • impossible is nothing - www.x2nie.com


But I found some bug made by me. Such as when a "begin" moved to the left (by deleting trailing space).
I will fix them later.
-----------------------------
You need to add invalidation calls.

Add or remove space in front of a "begin" (then resize the editor to invalidate)


I have bugfixed it in my SynEditMarkupFoldColors.pas . please update your download.
It also more simple than your suggetion, it doesn't require any list, nor checking what should be next paint.
Thanks for your LinesInWindow.   It was difficult for me to guess what is proc/prop name I need.


Anyway,
I think it would be better when the bugfix happens in TSynCustomFoldHighlighter (SynEditHighlighterFoldBase.pas),
Because, the job (invalidate rest line) has already been done (without my bugfix)
when I delete "b" from "begin",  Also, when I edit again the "egin" into "begin".


-------------
The only problem is the nested begin~end of procedure has similar color to begin~end of parent procedure.
It even annoying when the children procedure in several depth.


Try below code (copy + paste) into the Pascal Tab (or see attachment)



Code: Pascal  [Select]
  1.  
  2. procedure TSynEditMarkupFoldColors.DoTextChanged(StartLine, EndLine,
  3.   ACountDiff: Integer);
  4.  
  5.  
  6. var
  7.     y,i,LCnt : integer;
  8.  
  9.  
  10.   function GetPairCloseFold(aRow, X : integer  ): Integer;
  11.   var
  12.     y,i,LCnt : integer;
  13.     HL: TSynCustomFoldHighlighter;
  14.     NodeList: TLazSynFoldNodeInfoList;
  15.     TmpNode, CloseNode: TSynFoldNodeInfo;
  16.  
  17.  
  18.     function FindEndNode(StartNode: TSynFoldNodeInfo;
  19.                        {var} YIndex, NIndex: Integer): TSynFoldNodeInfo;
  20.       function SearchLine(ALineIdx: Integer; var ANodeIdx: Integer): TSynFoldNodeInfo;
  21.       begin
  22.         NodeList.Line := ALineIdx;
  23.         repeat
  24.           inc(ANodeIdx);
  25.           Result := NodeList[ANodeIdx];
  26.         until (sfaInvalid in Result.FoldAction)
  27.            or (Result.NestLvlEnd <= StartNode.NestLvlStart);
  28.       end;
  29.  
  30.  
  31.     begin
  32.       Result := SearchLine(YIndex, NIndex);
  33.       if not (sfaInvalid in Result.FoldAction) then
  34.         exit;
  35.  
  36.  
  37.       inc(YIndex);
  38.       while (YIndex < LCnt) and
  39.             (HL.FoldBlockMinLevel(YIndex, StartNode.FoldGroup, [sfbIncludeDisabled])
  40.              > StartNode.NestLvlStart)
  41.       do
  42.         inc(YIndex);
  43.       if YIndex = LCnt then
  44.         exit;
  45.  
  46.  
  47.       NIndex := -1;
  48.       Result := SearchLine(YIndex, NIndex);
  49.  
  50.  
  51.       if (Result.LogXEnd = 0) or (sfaLastLineClose in Result.FoldAction) then
  52.         Result.FoldAction := [sfaInvalid]; // LastLine closed Node(maybe force-closed?)
  53.     end;
  54.  
  55.  
  56.   var y2,i2 : integer;
  57.   begin
  58.     Result := -1;
  59.     y := aRow -1;
  60.  
  61.  
  62.     HL := TCustomSynEdit(self.SynEdit).Highlighter as TSynCustomFoldHighlighter;
  63.     HL.CurrentLines := Lines;
  64.     LCnt := Lines.Count;
  65.     HL.FoldNodeInfo[y].ClearFilter; // only needed once, in case the line was already used
  66.  
  67.  
  68.     NodeList := HL.FoldNodeInfo[y];
  69.     NodeList.AddReference;
  70.     try
  71.       NodeList.ActionFilter := [sfaOpen];
  72.       i := 0;
  73.       repeat
  74.         TmpNode := NodeList[i];
  75.  
  76.  
  77.         if TmpNode.LogXStart < X-1 then
  78.         begin
  79.           inc(i);
  80.           continue;
  81.         end;
  82.  
  83.  
  84.         //find till valid
  85.         while (sfaInvalid in TmpNode.FoldAction) and (i < NodeList.Count) do
  86.         begin
  87.           inc(i);
  88.           TmpNode := NodeList[i];
  89.         end;
  90.         if not (sfaInvalid in TmpNode.FoldAction) then
  91.         begin
  92.           CloseNode := FindEndNode(TmpNode, y, i);
  93.           //AddHighlight(TmpNode);
  94.           Result := CloseNode.LineIndex;
  95.           exit;
  96.         end;
  97.  
  98.  
  99.         inc(i);
  100.       until i >= NodeList.Count;
  101.  
  102.  
  103.     finally
  104.       NodeList.ReleaseReference;
  105.     end;
  106.   end;
  107.  
  108.  
  109.  
  110.  
  111.   function IsFoldMoved( aRow: Integer ): integer;
  112.   var S : string;
  113.     i,n : integer;
  114.   begin
  115.     Result := -1;
  116.     n := -1;
  117.  
  118.  
  119.     S := Caret.LineText;
  120.     for i := 1 to Min(Length(S), Length(FPrevCaretText)) do
  121.     begin
  122.       if S[i] <> FPrevCaretText[i] then
  123.       begin
  124.         n := i;
  125.         break;
  126.       end;
  127.     end;
  128.  
  129.  
  130.     if n < 0 then exit;
  131.  
  132.  
  133.     Result := GetPairCloseFold(aRow, n);
  134.     if Result > 0 then
  135.     begin
  136.       with TCustomSynEdit(SynEdit) do
  137.         Result := min(Result, TopLine +LinesInWindow);// . .RowToScreenRow(i);
  138.     end;
  139.  
  140.  
  141.   end;
  142. var
  143.   EndFoldLine,LineEnd,y : integer;
  144. begin
  145.   if EndLine < 0 then exit; //already refreshed by syn
  146.  
  147.  
  148.   y := Caret.LineBytePos.y;
  149.   EndFoldLine := IsFoldMoved(y);
  150.   if EndFoldLine > 0 then
  151.   begin
  152.     InvalidateSynLines(y+1, EndFoldLine);
  153.   end;
  154.  
  155.  
  156.   FPrevCaretText := Caret.LineText;
  157.   // I found that almost anything has been repaint by the SynEdit,
  158.   // except the trailing space editing: we should repaint them here.
  159.  
  160.  
  161. end;
  162.  
When you were logged in, you can see attachments.
Lazarus Trunk @ Windows7 64bit, XP 32bit, Debian under VirtualMachine

x2nie

  • Sr. Member
  • ****
  • Posts: 478
  • Impossible=I don't know the way
    • impossible is nothing - www.x2nie.com
Well, This SynEdit Fold Color is only works fine with TSynHighlighterPasSyn.
To make this useful for any other SynHighlighter,
i think we need to add some nice feature to TSynCustomFoldHighlighter  8-)


such easy way to find any pair of begin~end, or C++ "{"~"}" enabled by default.


In real world, this SynMarkup can help programmer to quickly identify the wrong logic caused by any mismatch of block-sign (such as missing block "end").
It's crucial feature when we work in complex nested block code such PHP, Pascal, and XML based code.
 
Let me describe this feature by pictures below.


-----
Ideally, TSynCustomFoldHighlighter (or TLazSynEditNestedFoldsList?) should able to provides any pair of opening+closing fold.
If it works, TSynDemoHlFold ( = simplest highlighter with folding ability) will be sexy too.

When you were logged in, you can see attachments.
Lazarus Trunk @ Windows7 64bit, XP 32bit, Debian under VirtualMachine

x2nie

  • Sr. Member
  • ****
  • Posts: 478
  • Impossible=I don't know the way
    • impossible is nothing - www.x2nie.com
if you have a problem with my demo app, here my blind solution:


SynEditHighlighterFoldBase.pas line #683
Code: Pascal  [Select]
  1. function TLazSynFoldNodeInfoList.Count: Integer;
  2. begin
  3.   //if not FValid then exit(-1);
  4.   if not FValid then exit(0); //x2nie
  5.  
  6.  
  7.   DoFilter(-1);
  8.   Result := FFilteredCount;
  9. end;
When you were logged in, you can see attachments.
Lazarus Trunk @ Windows7 64bit, XP 32bit, Debian under VirtualMachine

Basile-B

  • Sr. Member
  • ****
  • Posts: 457
    • GH...
In real world, this SynMarkup can help programmer to quickly identify the wrong logic caused by any mismatch of block-sign (such as missing block "end").
It's crucial feature when we work in complex nested block code such PHP, Pascal, and XML based code.

Does it work on the indentation rather than on open/close pairs ?

x2nie

  • Sr. Member
  • ****
  • Posts: 478
  • Impossible=I don't know the way
    • impossible is nothing - www.x2nie.com
In real world, this SynMarkup can help programmer to quickly identify the wrong logic caused by any mismatch of block-sign (such as missing block "end").
It's crucial feature when we work in complex nested block code such PHP, Pascal, and XML based code.

Does it work on the indentation rather than on open/close pairs ?
Yes and No.


Yes: the colouring aspect it self: will work in any TSynCustomFoldHighlighter,
no matter indentation (such Python) or paired block programming language (such pascal, c/c++).
Because it depends on folding system, not depends on ony programming syntax.


No: since indentation programming language (such as Python) never requires any ending-block notation,
so my TSynEditMarkupFoldColors wouldn't be able to help programmer finding missing ending-block (which is not existed).

----
To make this useful for any other SynHighlighter, i think we need to add some nice feature to TSynCustomFoldHighlighter such easy way to find any pair of begin~end, or C++ "{"~"}" enabled by default.
What I refer to begin~end, or C++ "{"~"}" is the X/Y location of that (act as opening folding & closing mark)
which are only required to locate it's position of drawing vertical lines.
So, it doesn't really mean require 'begin', 'end', '{', '}', nor anything programming language specific.


----
But unfortunately, current lazarus synhighlighter for python was not inherited from TSynEditMarkupFoldColors. So it doesn't work, but its not limitation of my class.
Even worst, it doesn't yet work with anything else either, only with TSynHighlighterPas currently works. :(

I wish in near future we can reach it really works on any TSynCustomFoldHighlighter, ... .
i think it happened because of development of lazarus's pascal editor was very far & fast, compared to development progress in SynEdit it self.
« Last Edit: December 04, 2015, 03:35:53 pm by x2nie »
When you were logged in, you can see attachments.
Lazarus Trunk @ Windows7 64bit, XP 32bit, Debian under VirtualMachine

Basile-B

  • Sr. Member
  • ****
  • Posts: 457
    • GH...
By the way, I suppose it only works on the master branch ? I just tried to add it to Coedit (which is based on latest stable release) and TSynEdit doesn't have a MarkupManager member... :'(

x2nie

  • Sr. Member
  • ****
  • Posts: 478
  • Impossible=I don't know the way
    • impossible is nothing - www.x2nie.com

Master branch? you might rever to git.
Yes, it require the trunk svn lazarus.
I added a public property in svn/trunk
SynEdit.MarkupManager

So you can drop a SynEdit on a form, subclass your own Markupclass, and play with it.


I will support Lazarus 1.4 too !!  8-)
but, maybe not today. :-X  Happy weekend !
When you were logged in, you can see attachments.
Lazarus Trunk @ Windows7 64bit, XP 32bit, Debian under VirtualMachine

Basile-B

  • Sr. Member
  • ****
  • Posts: 457
    • GH...
on 1.4, I've managed to add it like this (inside my TSynEdit sub class constructor):

Code: Pascal  [Select]
  1. fMarkupIndent:= TSynEditMarkupFoldColors.Create(self);
  2. TSynEditMarkupManager(MarkupMgr).AddMarkUp(fMarkupIndent, true);

but I get some range violation, line 3325 of SynEditFoldedView

Code: Pascal  [Select]
  1. SetLength(OpenIdx, FGroupCount, FFoldNodeInfoList.Count);

Edson

  • Hero Member
  • *****
  • Posts: 1054
Well, This SynEdit Fold Color is only works fine with TSynHighlighterPasSyn.
To make this useful for any other SynHighlighter,
i think we need to add some nice feature to TSynCustomFoldHighlighter  8-)

such easy way to find any pair of begin~end, or C++ "{"~"}" enabled by default.

Well, I felt sad when discovered "TSynCustomFoldHighlighter" has not complete information about the fold levels, when I was designing my highlighter: http://forum.lazarus.freepascal.org/index.php/topic,21727.msg145713.html#msg145713
Lazarus 1.6 - FPC 3.0.0 - x86_64-win64 on  Windows 7

Martin_fr

  • Administrator
  • Hero Member
  • *
  • Posts: 5691
    • wiki
It will take a while to answer all this..... so please be patience.

Quote
I think it would be better when the bugfix happens in TSynCustomFoldHighlighter (SynEditHighlighterFoldBase.pas),
Because, the job (invalidate rest line) has already been done (without my bugfix)
when I delete "b" from "begin",  Also, when I edit again the "egin" into "begin".

The HL does indeed some of the invalidation. If a begin is inserted/removed it will invalidate down to the last affected end (potentially end of file).

But: if a "begin" simply changes x pos (add a space on that line, but somewhere before the begin) then for the HL no invalidation is needed. SynEdit will invalidate just that line (sometimes 1 or 2 extra)

Both of the above are as intended.
Code: Pascal  [Select]
  1. ...
  2.    if a then Begin // foo
  3.              |
  4.              |
  5.    end
  6.  
- if you edit "a" to "val" then the horizontal line needs to indent.
- if you edit "foo", then it does not change the line.

So only your code can know if an edit in a code-line changes the vertical lines. You then need to know how many lines below need to be invalidated.
Code: Pascal  [Select]
  1. ...
  2.    if a then Begin while b do begin// foo
  3.              |                |
  4.              |                |
  5.              |       end
  6.              |
  7.              |
  8.    end
  9.  

Code: Pascal  [Select]
  1. The only problem is the nested begin~end of procedure has similar color to begin~end of parent procedure.
I look at it later.
Including disabled nodes, and you should get them in the list, but you need to do your own count. (I mentioned before, I can add more details, but later)

Quote
Well, This SynEdit Fold Color is only works fine with TSynHighlighterPasSyn.
To make this useful for any other SynHighlighter,
i think we need to add some nice feature to TSynCustomFoldHighlighter
Yep, some code needs to be factored up from pas to the base class.

Again later. sorry , busy.

For this we also need to look at foldconfig. And that will also mean you get the ability to configure which nodes to highlight and which not.

Quote
It's crucial feature
agreed, and I try to allocate as much of my time as I can.

Quote
Code: Pascal  [Select]
  1. function TLazSynFoldNodeInfoList.Count: Integer;
  2. begin
  3.   //if not FValid then exit(-1);
  4.   if not FValid then exit(0); //x2nie
I need to check if -1 is used anywhere....


-------------
I will add more asap