Recent

Author Topic: TSynEdit and indentation folding  (Read 568 times)

rufusROFLpunch

  • New member
  • *
  • Posts: 6
TSynEdit and indentation folding
« on: April 06, 2020, 05:36:31 pm »
I'm attempting to write a highlighter with folding for the Nim language. I've run into an issue with the indentation-based folding. The problem I am have is that the folding engine seems to expect a token on the line where folding begins, but with the indentation, it's the line after the where the fold begins. This is probably hard to understand, let me try and demonstrate.

For a C-based language:

Code: [Select]
int main() {
    // some stuff
}

The folded section begins at the line with the opening brace. But with Nim, the token is actually on the following line:

Code: [Select]
type
    SomeObj = object
        name: string

So, it should fold on the type line, for instance, but the token that says whether it should fold is the leading whitespace at the beginning of the line. When I lex line 1, I store the indentation level in the range. Then when I lex line 2, I compare the indentation level from the range to the current line's indentation level. If it's greater than the previous line's indentation, I can initiate a fold using `StarCodeFoldingBlock` however that's starts folding on the current line, I actually need the fold to start on line 1.

I hope that makes sense. If anyone knows of a way to apply the fold to the PREVIOUS line, or even a better way to do this, I would appreciate the advice.

Martin_fr

  • Administrator
  • Hero Member
  • *
  • Posts: 6368
    • wiki
Re: TSynEdit and indentation folding
« Reply #1 on: April 06, 2020, 06:05:22 pm »
Making your example a bit longer....
Code: Text  [Select][+][-]
  1. type
  2.     SomeObj = object
  3.         name: string
  4.         foo: string
  5.     Other = object
  6.         bar: string
  7. var
  8.     abc: foo
  9.  
If I understand you, there should be a [+-] on each of the yellow lines?

When folding "type", it should look like
Code: Text  [Select][+][-]
  1. type   [...]
  2. var
  3.     abc: foo
  4.  

In Pascal we have that same issue, but for ending some fold blocks. A "var" after a "type" section, closes the type section on the line before the "var".

From the back of my head.....

You have several things to look at.

FoldBlockMinLevel / FoldBlockEndLevel
You can probably increase/decrease the internal counters (if you keep them) on the line that has the indent.
Then the getter can look one line ahead (for the "end" level).
Because if you have the "virtual" fold opening on the "type" line, then at the end of that line, you have 1 level open (but you store it for the next line only).
Probably similar for closing the fold.....


procedure InitFoldNodeInfo.
This scans the actual line, and returns the token.
Afaik (needs testing) a token can be empty (i.e. the end of line)

Look at ScanFoldNodeInfo in the pas HL, it adds virtual close nodes for fold-close from the next line.
I would expect the same is possible for adding empty opening nodes at the end of the line. (based on FoldBlockMinLevel / FoldBlockEndLevel data)

The blocks are empty because fStringLen = 0
See DoInitNode Called by EndFoldBlock.



It's quite a lot of code to go through. Feel free to ask for more info


Martin_fr

  • Administrator
  • Hero Member
  • *
  • Posts: 6368
    • wiki
Re: TSynEdit and indentation folding
« Reply #2 on: April 06, 2020, 06:11:02 pm »
In Pascal you can setup "click on the 'begin' token" to fold.  (instead of clicking the [+-].

That would not be possible. You can not click the whitespace indent to fold on the line above.
You only get the [+-]

rufusROFLpunch

  • New member
  • *
  • Posts: 6
Re: TSynEdit and indentation folding
« Reply #3 on: April 06, 2020, 08:48:17 pm »
Martin, thanks for your very helpful reply. It led me on the path I needed. I think I got it resolved. Here's what I ended up doing... Originally, I was setting the indentation level in the Next method. Any whitespace at the beginning of the line only was treated as a separate tkIndentation token, and I was saving the indentation level with the range, which is when I ran into the problem described above.

However, after reading your messages and digging into the methods and properties you suggested I look at, I found a different approach. I decided to look at the indentation in SetLine instead, and do a lookahead to the next non-blank line for that indentation level, and use that to determine whether to set the fold or not. Here is the full method I ended up using:

Code: Pascal  [Select][+][-]
  1. procedure TSynNimSyn.SetLine(const NewValue: string; LineNumber: integer);
  2. var
  3.   currentIndent: integer;
  4.   nonBlankIndex: integer;
  5.   nextIndent: integer;
  6. begin
  7.   inherited SetLine(NewValue, LineNumber);
  8.   FLine := NewValue;
  9.   FLineNo := LineNumber;
  10.   FTokenStart := 1;
  11.   FTokenEnd := 1;
  12.   FIndent := 0;
  13.   Next;
  14.  
  15.   nonBlankIndex := NextNonBlankLineIndex; // Find the next line which is not just whitespace
  16.   if ((FLineNo + 1) < CurrentLines.Count) and (nonBlankIndex > -1) then
  17.   begin
  18.     currentIndent := GetStringIndentation(FLine);
  19.     nextIndent := GetStringIndentation(CurrentLines[nonBlankIndex]);
  20.  
  21.     if nextIndent = 0 then
  22.       while (CodeFoldRange.CodeFoldStackSize > 0) do
  23.         CodeFoldRange.Pop(True)
  24.     else if (nextIndent > currentIndent) and (FLine.Length > 0) then
  25.       StartCodeFoldBlock(nil)
  26.     else if nextIndent < currentIndent then
  27.       EndCodeFoldBlock;
  28.   end
  29.   else
  30.     while (CodeFoldRange.CodeFoldStackSize > 0) do
  31.       CodeFoldRange.Pop(True);
  32. end;
  33.  

To summarize, when a new line is set on the highligher, I get the new line's indentation, and the next (non-blank) line's indentation. If the next non-blank indentation level is more than the current line, I start a code fold block. If the next (non-blank) line's indentation is less than the current line, I decrease the fold level. If the next non-blank line has no indentation, or if there are no more non-blank lines, pop all of the remaining indentation levels off the stack.

There are probably some corner cases I haven't found yet, but seems to be doing what I was looking for.

Martin_fr

  • Administrator
  • Hero Member
  • *
  • Posts: 6368
    • wiki
Re: TSynEdit and indentation folding
« Reply #4 on: April 07, 2020, 02:46:10 am »
The only issue with look-ahead is, that you need to invalidate lines backwards....

Example:
Code: Text  [Select][+][-]
  1.     foo
  2.         bar
  3.         123
If you remove the spaces in "bar", then you need to update the line above.

The Pascal HL, does that for one line. So you can check how it does (Late now, so not doing that myself, but ask and I do tomorrow).

Since you skip empty lines, you may need to go back further.

--
Same if spaces are added.


I cant think of a better way (than scan ahead) though.

Even if you could store info on the line with the spaces, you would need to go back, and find the line for which to invalidate painting. (since a [+-] might need to be painted).

If you did not skip empty lines, maybe with some efforts, it would work with just invalidating/painting the extra line above..... But not sure, and with empty line skip.....
« Last Edit: April 07, 2020, 02:48:33 am by Martin_fr »

rufusROFLpunch

  • New member
  • *
  • Posts: 6
Re: TSynEdit and indentation folding
« Reply #5 on: April 08, 2020, 01:50:45 am »
I have been at it for the last day and can’t seem to figure out how to invalidate previous lines. The logic of the Pascal highlighter is quite complicated.

Martin_fr

  • Administrator
  • Hero Member
  • *
  • Posts: 6368
    • wiki
Re: TSynEdit and indentation folding
« Reply #6 on: April 08, 2020, 02:29:08 am »
Ok took me a while. It actually is a hack / hardcoded.

Code: Pascal  [Select][+][-]
  1. procedure TSynCustomHighlighter.ScanRanges;
  2. ..
  3. CurrentLines.SendHighlightChanged(StartIndex - 1, EndIndex - StartIndex + 1)

That is PasHl does not rescan the line above. IIRC It stores info (for Fold...Lever) on the next line, and if the HL is ask for info on a line, it checks the info stored on the next line.

"rescan" here means: in ScanRanges.
Each line is also scanned for other reasons, such as painting, or the fold-provider asking to scan a line (which IIRC is only done if Fold...Level, indicate a need for that)

But the HL tells SynEdit to invalidate/paint one line above the scan begin => and thus the fold provider re-evals that line....
(that is the code above, sending the invalidated range)


But you can subclass CurrentRanges, and when it sets the first line needed to be scanned => to what you need.
That does mean a lot more scanning....

Better, find a way to store info, so that the main-scan does not need to redo the lines above.
Override PerformScan.
The you can do several things:
- At the end of PerformScan => Send your own invalidate range. (so lines above will be repainted if needed).
  Sending the message twice is harmless, so don't worry that part of it will be sent again.

- You can either go back at the start of PerformScan, and find your ideal startline. But better not.
- Or: While you scan a line, check if the fold info on the previous lines "range" is correct. And only if not, then iterate back over those ranges and update them (you probably do not need to scan those lines for this, just access the ranges)
When you go back, keep the info, as this is what you need to invalidate for painting.....

Note, if you update a range, you must copy it, and update the copy.
Then stare it with "fRanges.GetEqual(FCodeFoldRange);"
See GetRange and SetRange.
One and the same range object is used for many lines. So if you update it without those steps, you change a lot more than you wanted....
"getrange" actually has some (old) documentation) (in the xml docs / fpdoc)

Hope that helps
« Last Edit: April 08, 2020, 02:31:20 am by Martin_fr »

rufusROFLpunch

  • New member
  • *
  • Posts: 6
Re: TSynEdit and indentation folding
« Reply #7 on: April 08, 2020, 04:06:42 am »
Overriding PerformScan did the trick! As a quick test, I overrode it always start from index 0 (first line of file). Obviously not the most optimized method. I can probably tweak it but just having it backtrack to the last top level line (0 indentation) and scan forward from there. Thanks for the help, Martin! After reading through all the threads on the wiki page for syntax highlighting, you've really put the time in over the years to helping people.

Btw, I noticed you said this was not the preferred method. Is there a reason for that?

Martin_fr

  • Administrator
  • Hero Member
  • *
  • Posts: 6368
    • wiki
Re: TSynEdit and indentation folding
« Reply #8 on: April 08, 2020, 02:00:35 pm »
"preferred"...

The HL is supposed to be fast, and work with big files (millions of lines, if need).

Scanning the entire file each time (i.e. on each key press) can have an impact. (Also, in case your CPU is fast enough, there are laptop batteries)


Therefore to always go back in PerformScan is not a good idea. (PerformScan is a good place, just the "always" is not)
You should only go back, if indeed required. And only as far as needed.

And because your only need to pay attention to the indent, you do not need to scan the rest of those backwards line. You could do a cheaper scan.
Though that is an optimization, that may not be worth the cost of implementation....



Btw the current implementation has a flaw. It always scans to the bottom, if top level fold changes. (i.e. in pascal, if you open a comment, that does not close). It only needs to scan to the end of visible area. (Though that will change if ever there is a mini map....)

 

TinyPortal © 2005-2018