Recent

Author Topic: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}  (Read 111300 times)

Martin_fr

  • Administrator
  • Hero Member
  • *
  • Posts: 5695
    • wiki
Quote from: martin_fr
So only your code can know if an edit in a code-line changes the vertical lines. You then need to know how many lines below need to be invalidated.

You probably do not need to store the height of any vertical line. Since any change to that will lead to invalidation by the HL or not need full redraw.

But you need to know when a vertical line moves x pos.

x2nie

  • Sr. Member
  • ****
  • Posts: 478
  • Impossible=I don't know the way
    • impossible is nothing - www.x2nie.com
You probably do not need to store the height of any vertical line. ...
But you need to know when a vertical line moves x pos.
I did it. I've able to identify when a 'begin' is moved or not, by comparing the both texts :
* text on the line/caret before editing
* text that has been updated.
It's done because:
+ when the Caret moved up or down, I store the text in the line.
+ when text changed, I can only get the text being changed, but I already have text before change that I store when caret moved vertical.




Quote
I think it would be better when the bugfix happens in TSynCustomFoldHighlighter (SynEditHighlighterFoldBase.pas),
Because, the job (invalidate rest line) has already been done (without my bugfix)
when I delete "b" from "begin",  Also, when I edit again the "egin" into "begin".

The HL does indeed some of the invalidation. If a begin is inserted/removed it will invalidate down to the last affected end (potentially end of file).

But: if a "begin" simply changes x pos (add a space on that line, but somewhere before the begin) then for the HL no invalidation is needed. SynEdit will invalidate just that line (sometimes 1 or 2 extra)

Both of the above are as intended.
if my bugfixed invalidation runs, so the rest lines possibly invalidated twices !
Can it configured outside SynMarkup? As you said, people play in laptop, and every invalidate cost battery ha ha haa... >:D




Quote
Quote
Well, This SynEdit Fold Color is only works fine with TSynHighlighterPasSyn.
To make this useful for any other SynHighlighter,
i think we need to add some nice feature to TSynCustomFoldHighlighter
Yep, some code needs to be factored up from pas to the base class.


Quote
It's crucial feature
agreed, and I try to allocate as much of my time as I can.
Thanks you!


----
But the position of 'begin'~'end' (or '{' and '}' in java/c) is not for drawing vertical purpose only.
Another reason we need it (knowing their coordinate) is also for drawing that pair in same color as vertical lines.
and in SynMarkup, drawing text in nested color require the position of those elements.


____
My bottom line is: how to make Lazarus FoldHL (Demo) multi level color folding with my SynMarkup class,
by a little effort.




----
My next plan is to draw horizontal dotted line when the left position of begin & end is horizontally different.
But, before do that, I should able to detect the deferential between begin & end, from up to bottom
I think the easiest way is when HL puts this info on StartCodeFoldBlock & EndCodeFoldBlock.

But, Maybe it couldn't be done automagically, because each HL has different way to parse, hence different possibility of when to callStartCodeFoldBlock & EndCodeFoldBlock (can't make sure if a new token has already stored or not yet). Hence, the TSynCustomFoldHighlighter couldn't make sure where the token starts nor when it ends.


So, maybe  giving the each HL a chance to tell TSynCustomFoldHighlighter of where is the opening fold & closing fold located, is perfect solution.
When the open~close token attached to a range, we can easily detect the 'end' coordinate when drawing the row of 'begin'.
 :)
Code: Pascal  [Select]
  1.  
  2. procedure TSynDemoHlFold.Next;
  3. begin
  4.   inherited Next;
  5.   if (copy(FLineText, FTokenPos, FTokenEnd - FTokenPos) = '-(-') then
  6.   begin  
  7.      StartCodeFoldBlock(nil);
  8.  
  9.      // store the opening fold bracket after push / attached new child-area
  10.      StartCodeFoldToken( FTokenPos, FTokenEnd ); // if level increased
  11.  
  12.   end;
  13.  
  14.   if (copy(FLineText, FTokenPos, FTokenEnd - FTokenPos) = '-)-') then
  15.   begin  
  16.     // put the closing fold bracket before pop / detached, back to parent work-area
  17.      EndCodeFoldToken( FTokenPos, FTokenEnd );  // level decreasing.
  18.  
  19.      EndCodeFoldBlock;(
  20.   end;
  21. end;
« Last Edit: December 04, 2015, 07:12:43 pm by x2nie »
When you were logged in, you can see attachments.
Lazarus Trunk @ Windows7 64bit, XP 32bit, Debian under VirtualMachine

Martin_fr

  • Administrator
  • Hero Member
  • *
  • Posts: 5695
    • wiki
Quote
I think the easiest way is when HL puts this info on StartCodeFoldBlock & EndCodeFoldBlock.
Yes that code needs to move up to the base class.

The important things (and just from top of my head, there may be more).

FoldInfo is only collected if FCatchNodeInfo is true.
That is when TSynPasSyn.InitFoldNodeInfo is called.

This affects SetLine !!!!! And I need to look into detail why, but it will be important.
So if FCatchNodeInfo  moves up, it must have a protected readable property.

InitFoldNodeInfo must also move up, but parts are pas specific.

FOLDGROUP_* is only pascal (it is for ifdef folds). Other HL may later get that too, but different issue / later.

It isnt going to be easy to move, because some of the flags sfaOpen ... are pas specific.

Some come from fold config. Or depend on it.

For any HL to use this, they need fold config.

Also foldconfig should get a new flag sfaMarkupOutline, that will enable/disable outlining (your markup would check for the presence of it)

There may be a 2nd flag needed to help changing colors on nested procs (you still need to do counting in the markup, you can not use the levels directly)

---------------------------
I get more on it later.


You could start by adding a copy to your demo hl, then get the understanding, and that will later help deciding on how to refactor

Martin_fr

  • Administrator
  • Hero Member
  • *
  • Posts: 5695
    • wiki
DoCaretChanged for invalidate is not enough.

text can change by codetool and other mean (2 shared editors, text edited by one changes in both). then the caret does not move.

Martin_fr

  • Administrator
  • Hero Member
  • *
  • Posts: 5695
    • wiki
OK, 2 bits of info.

1) foldconfig.

currently length is set in .Create. not good, maybe move.

GetFoldConfigInstance
gets the initial setting.

for each foldtype (cfbtBeginEnd,...) it indicates, if it can
Code: Pascal  [Select]
  1. TSynCustomFoldConfigMode = (fmFold, fmHide, fmMarkup);
- fold
- hide
- markup (word triplet at caret)

you could add fmMarkupOutline then this can be configured, and the HL can indicate which nodes are for your module.

the HL can set 2 flags (maybe more)
- sfaMarkupOutline
  visible outline
- sfaMarkupOutlineIndend
  not outlined, but increase color (e.g. a nested procedure block)

those would then be set in HL.InitNode() according to the language rules.

As for the "level" of intend. The HL levels are not correct, but when you go through Nest : TLazSynEditNestedFoldsList; you could keep count of nodes that have the flag (you need to start left to right then).

Or simply set them on the result FHighlights. But then you need to add dummy entries for the invisible sfaMarkupOutlineIndend. So that is not desirable.

-------------------------------
As for adding fold info to the base class.

PasHl has a lot of custom code. I wouldnt move it just yet.
Add virtual methods to the base class with whatever generic code is possible, and keep them overriden in pasHl

Then you can start getting code for the other HL.

It is important to collect node info only when requested. (per line). So it does not slow down normal hl scans.
 

x2nie

  • Sr. Member
  • ****
  • Posts: 478
  • Impossible=I don't know the way
    • impossible is nothing - www.x2nie.com
PasHl has a lot of custom code. I wouldnt move it just yet.
Add virtual methods to the base class with whatever generic code is possible, ....

It is important to collect node info only when requested. (per line). So it does not slow down normal hl scans.

Then you can start getting code for the other HL.
All above is done. I have created TSynColorFoldHighlighter that can be used by other HL as base class for quick creating   HL with folding+ nested color ability. Yes, it harvest the node info only when requested, so it is efficient.


For demo, I created TSynHighlighterBracket = class (TSynColorFoldHighlighter)
it works gracefully. (see attachment)

I didn't synchro the the pasHL yet, so it still:
Code: Pascal  [Select]
  1. TSynPasSyn               = class(TSynCustomFoldHighlighter)
  2. TSynColorFoldHighlighter = class(TSynCustomFoldHighlighter)
  3. TSynHighlighterBracket   = class(TSynColorFoldHighlighter)
  4.  


-----
Yesterday and before, I experimented with adding properties directly to TSynCustomHighlighterRange.
It was more efficient because doesn't need to re-scan, no recalculation needed.
It also been worked, but it was unstable somehow. When I scroll the SynEdit, colors changed unpredictably.
Days and night curiosity not paid. It was nightmare for these days.  >:D
I ignored those all changes and create a whole new middle class "TSynColorFoldHighlighter" and today was beautiful day.  8-)


-----
Now the problem is TLazSynEditNestedFoldsList doesn't work at all. It is a bit mysterious  for my brain.
Any clue ?


-----
(I am studying your last post.)
.
« Last Edit: December 08, 2015, 08:27:59 am by x2nie »
When you were logged in, you can see attachments.
Lazarus Trunk @ Windows7 64bit, XP 32bit, Debian under VirtualMachine

Martin_fr

  • Administrator
  • Hero Member
  • *
  • Posts: 5695
    • wiki
All above is done. I have created TSynColorFoldHighlighter that can be used by other HL as base class for quick creating   HL with folding+ nested color ability. Yes, it harvest the node info only when requested, so it is efficient.

I look at this later. The aim should be to patch the TSynCustomFoldHighlighter.

Quote
Yesterday and before, I experimented with adding properties directly to TSynCustomHighlighterRange.
It was more efficient because doesn't need to re-scan, no recalculation needed.
Not sure what you did, but the range (and all data/classes held by it) must be immutable.

The exception is the one used to work on the current line.
See SetRange(), GetRange() they make copies/assign.
New fields must be in the compare method.

But new fields should be kept to a minimum. I dont think the color level is needed. It is easy enough to calculate.

I just started wondering if the existing FPasFold...Level are really needed. But that is not part of this issue here.

Quote
Now the problem is TLazSynEditNestedFoldsList doesn't work at all. It is a bit mysterious  for my brain.
Any clue ?

what exactly goes wrong?


One way would be to look at SourceSynEditor, and copy TIDESynGutterDebugHL
It lets you spy on your data (modify it / edit the Paint() to show what you need.), then you can see if all values in the HL are as expected


for the bracket fold hl , I get an assert
TLazSynEditNestedFoldsList.InitNestInfoForIndex
Code: Pascal  [Select]
  1.   assert(nfeHasHNode in FNestInfo[AnIndex].FFLags, 'nfeHasHNode in FNestInfo[AnIndex].FFLags');
  2.  

looks like it fails here
Code: Pascal  [Select]
  1.       if l >= GrpCnt[t] then continue;
  2.  

maybe AHighlighter.FoldBlockEndLevel()

x2nie

  • Sr. Member
  • ****
  • Posts: 478
  • Impossible=I don't know the way
    • impossible is nothing - www.x2nie.com
.. TSynColorFoldHighlighter  can be used by other HL as base class
for quick creating   HL with folding+ nested color ability. Yes, it harvest the node info only when requested, so it is efficient.
.. The aim should be to patch the TSynCustomFoldHighlighter.
Owh? okay, I think that is good idea. 8)
it would be someday, when the TSynPasSyn was redesigned too, to reflect that changes.
But now, while lazarus deeply requires the stable TsynPasSyn, I can not do that.


Quote
Now the problem is TLazSynEditNestedFoldsList doesn't work at all. It is a bit mysterious  for my brain.
Any clue ?
what exactly goes wrong?
The vertical lines (generated by nested fold info) aren't painted. I guess it will be solved by following your (last post) guidance.


I still trying
« Last Edit: December 09, 2015, 06:42:55 am by x2nie »
When you were logged in, you can see attachments.
Lazarus Trunk @ Windows7 64bit, XP 32bit, Debian under VirtualMachine

Martin_fr

  • Administrator
  • Hero Member
  • *
  • Posts: 5695
    • wiki

The vertical lines (generated by nested fold info) aren't painted. I guess it will be solved by following your (last post) guidance.

I didnt debug it to the end, but I noticed that a loop in TLazSynEditNestedFoldsList.InitNestInfoForIndex( exited early
Quote
looks like it fails here
Code: Pascal  [Select]
  1.       if l >= GrpCnt[t] then continue;
  2.  

maybe AHighlighter.FoldBlockEndLevel()

I then get the assert
Code: Pascal  [Select]
  1.   assert(AnIndex >= FEvaluationIndex, 'TLazSynEditNestedFoldsList.InitNestInfoForIndex Index not found');

A bit of background (you need to know fold levels, see wiki, but I guess you know them by now

TLazSynEditNestedFoldsList looks for nodes that opened in a previous line AND are still open in the requested line.

GrpCnt[t] is the foldlevel from the requested line. (numbers may be off by one, if it is end vs start of fold.
Code: Pascal  [Select]
  1. begin  // level 0
  2.   begin    // level 1 - not of interest, will close before requested line
  3.   end  
  4.   // requested line/ level 1
  5. end
  6.  
the example is not real, it would actually require several folds starting on the same line, but ignore....


when debuginh the  "-(-"  HL, GrpCnt[t] is always 0, that is wrong, it is set in InitSubGroupEndLevels

so I guess AHighlighter.FoldBlockEndLevel()  is not working.

--------------
GrpCnt = GroupCount , count for group ....

ingnore group, except for pas there is only one, with index 0 (zero).

in pascal there are 2 more groups for IFDEF and %REGION, because they can overlap with begin/end and each other, so they need there own count.


Martin_fr

  • Administrator
  • Hero Member
  • *
  • Posts: 5695
    • wiki
FoldBlockEndLevel may not know about the "pas" level, that you copied from the pas hl.

so you need to copy more code.

on the other hand, I suggest: drop the pas level again.

the levels in the HL are not always correct for your purpose of color picking. and even if they are now, they may change....

see my earlier mail about counting while going through the list.

x2nie

  • Sr. Member
  • ****
  • Posts: 478
  • Impossible=I don't know the way
    • impossible is nothing - www.x2nie.com
 :'( I am now completely confused. (but not frustrated).  :D  I don't really know what I have copied from SynPas  :-[ .




Yeah, Good advice, I think I need to restart from the very beginning.
For honest, actually I did only read a few of your earlier posts. I will read them all by now on.


-----
Anyway, as suggested, I've copy the debug gutter, so I can see the visible levels for each line.
(see attachmeng) Thanks you.
Therefor,  I am sure I have missed something in the middle work.


I will back when get something else.
When you were logged in, you can see attachments.
Lazarus Trunk @ Windows7 64bit, XP 32bit, Debian under VirtualMachine

x2nie

  • Sr. Member
  • ****
  • Posts: 478
  • Impossible=I don't know the way
    • impossible is nothing - www.x2nie.com

I have read wiki and many topic at this forum related to folding-synedit.

Good news: the nested-level is now back on track. Everything is good.
I guess, The bug was wrong value of FoldGroup:
Code: Pascal  [Select]
  1. node.FoldGroup := 0; // <--- wrong. the bug.
  2. node.FoldGroup := 1;//FOLDGROUP_PASCAL; //correct somehow


---
when debuginh the  "-(-"  HL, GrpCnt[t] is always 0, that is wrong, it is set in InitSubGroupEndLevels
so I guess AHighlighter.FoldBlockEndLevel()  is not working.
It is now too complicated for me to support both old and new classes.
I will drop the FoldHL.pas soon, I have drop it, because and SynHighlighterBracket is intended as replacement of that demo.


on the other hand, I suggest: drop the pas level again.

the levels in the HL are not always correct for your purpose of color picking. and even if they are now, they may change....
I am ready now. What is exactly the idea of "drop the pas level"?
does it means erasing below prop/proc from my TSynColorFoldHighlighterRange:
Code: Pascal  [Select]
  1.  
  2.     property PasFoldEndLevel: Smallint read FPasFoldEndLevel write FPasFoldEndLevel;
  3.     property PasFoldFixLevel: Smallint read FPasFoldFixLevel write FPasFoldFixLevel;
  4.     property PasFoldMinLevel: Smallint read FPasFoldMinLevel write FPasFoldMinLevel;
  5.  
  6.     function Add(ABlockType: Pointer = nil; IncreaseLevel: Boolean = True):
  7.         TSynCustomCodeFoldBlock; override;
  8.     procedure Pop(DecreaseLevel: Boolean = True); override;
  9.     procedure Clear; override;
  10.     function Compare(Range: TSynCustomHighlighterRange): integer; override;
  11.     procedure Assign(Src: TSynCustomHighlighterRange); override;
?
Above properties + methods is the heart of my nested-color-folding. without them, the color of opening tag cant be equal to the color of closing tag.


« Last Edit: December 09, 2015, 03:50:57 pm by x2nie »
When you were logged in, you can see attachments.
Lazarus Trunk @ Windows7 64bit, XP 32bit, Debian under VirtualMachine

x2nie

  • Sr. Member
  • ****
  • Posts: 478
  • Impossible=I don't know the way
    • impossible is nothing - www.x2nie.com
see my earlier mail about counting while going through the list.
I didn't understand of what and how of that counting...  . Would you like to be more specific?
When you were logged in, you can see attachments.
Lazarus Trunk @ Windows7 64bit, XP 32bit, Debian under VirtualMachine

x2nie

  • Sr. Member
  • ****
  • Posts: 478
  • Impossible=I don't know the way
    • impossible is nothing - www.x2nie.com
Well, This SynEdit Fold Color is only works fine with TSynHighlighterPasSyn.
To make this useful for any other SynHighlighter,
i think we need to add some nice feature to TSynCustomFoldHighlighter  8-)

such easy way to find any pair of begin~end, or C++ "{"~"}" enabled by default.

Well, I felt sad when discovered "TSynCustomFoldHighlighter" has not complete information about the fold levels, when I was designing my highlighter: http://forum.lazarus.freepascal.org/index.php/topic,21727.msg145713.html#msg145713


@Edson, can we continue discussing that here? if you still want to design a folding-highliter.
Fold Level SynHL was very hard to create, I am sure because I have tried it my self.
but it seem to be (nearly) easier nowadays.
What capabilities was your expected HL?


@A-M (the topic's author), hey you are disappearing when I were coming. it seem that I is you ha ha ha... No,
we are different person. what SynEditHighlighter you need? only pascal, eh?
 
When you were logged in, you can see attachments.
Lazarus Trunk @ Windows7 64bit, XP 32bit, Debian under VirtualMachine

Edson

  • Hero Member
  • *****
  • Posts: 1055
@Edson, can we continue discussing that here? if you still want to design a folding-highliter.
Fold Level SynHL was very hard to create, I am sure because I have tried it my self.
but it seem to be (nearly) easier nowadays.
What capabilities was your expected HL?

Hi @x2nie. I have already designed a folding-highlighter: https://github.com/t-edson/SynFacilSyn
In it, the support for folding is not as good as I wanted, mainly because of some limitation I found in TSynCustomFoldHighlighter.
Summarizing, I wanted this class understood about block of syntax, but it only knows about showing folding marks on the gutter > >:(.
I think this limitation can affect you too, on your development, because I have the feeling (no time to analyze all the posts  :-X) the feature you are implementing need to know about the syntax of the language.

Maybe it's better open a new thread "Modernizing the TSynCustomFoldHighlighter class"  :-\.
« Last Edit: December 09, 2015, 05:32:39 pm by Edson »
Lazarus 1.6 - FPC 3.0.0 - x86_64-win64 on  Windows 7