Lazarus

Programming => Packages and Libraries => SynEdit => Topic started by: Edson on August 07, 2013, 11:23:44 pm

Title: Folding in SynEdit
Post by: Edson on August 07, 2013, 11:23:44 pm
Hi,

I was implementing the "Folding" on a simple Editor, and reviewing the code of TSynLFMSyn, i noted the use of "CodeFoldRange" property. And I have some questions:

* What is "CodeFoldRange" for?
* Is there some kind of conflict on using ranges for Folding and ranges for Context Highlighter?

Thanks on advance.

PD: I'm using Lazarus 1.0.8 on i386/Win32.
Title: Re: Folding in SynEdit
Post by: Tristan32 on August 07, 2013, 11:34:08 pm
Don't know if this will help, but found this in here - https://github.com/alrieckert/lazarus/blob/master/components/synedit/synedithighlighterfoldbase.pas Line 768

Code: [Select]
// FCodeFoldRange is the working range and changed steadily
  // => return a fixed copy of the current CodeFoldRange instance,
  // that can be stored by other classes (e.g. TSynEdit)
Title: Re: Folding in SynEdit
Post by: Martin_fr on August 08, 2013, 09:43:58 am
the codefoldrange is a subclass of the normal range.

"range" represents the state of the scanner. the state is stored at the end of each line, so it can be used to continue on the next line.

"range" objects are immutable. the same object is used for all lines that end with the same state. (if the state changes (line edited), then another object is used, because the existing one must not be changed).

codefoldrange stores the nesting depth and type of fold. it also offers storage for the plain "range" info.

not sure if you can avoid it, since you are expected to inherit the highlighter from  TSynCustomFoldHighlighter.

http://wiki.lazarus.freepascal.org/SynEdit_Highlighter

----
Edit:

a range can be mutated if it is not stored yet.

so a working copy may be mutable
Title: Re: Folding in SynEdit
Post by: Edson on August 10, 2013, 12:11:05 am
Thanks for the responses.

I've implemented a simple folding, and it works. But I want to understand the philosophy of the folding, because I'm preparing a documentation.

Martin, when you say "... the normal range", Are you refering to the "TSynCustomHighlighterRange" class?.

Are the functions GetRange(), SetRange() expecting to work with a pointer to "TSynCustomHighlighterRange"?

I feel a little confused.


Thanks.
Title: Re: Folding in SynEdit
Post by: Martin_fr on August 10, 2013, 08:48:01 am
I do not know who choose the name "range". probably that was done in the original SynEdit, before it was forked for lazarus.

"range" can be an integer or enum/set (cast to a pointer), or an object (does not have to be TSynCustomHighlighterRange, but I haven't seen any other class being used)

see also the link from my last post.
http://wiki.lazarus.freepascal.org/SynEdit_Highlighter#Step_2:_Using_Ranges
uses an integer.
Title: Re: Folding in SynEdit
Post by: Edson on August 14, 2013, 02:17:36 am
I'm understanding.

Someone designed the manage of ranges through pointers, in order to have the ability for using objects (although we prefer to use enumerated, integers or sets).

But, In which cases it's useful to use objects as ranges?

Thanks.
Title: Re: Folding in SynEdit
Post by: Martin_fr on August 14, 2013, 07:55:15 am
But, In which cases it's useful to use objects as ranges?

1) when the data doesn't fit into 32 bit (assuming your app should run on 32 bit cpu too)

2) folding, because the baseclass does.
Title: Re: Folding in SynEdit
Post by: Edson on August 14, 2013, 06:33:45 pm
It's clear. Thanks Martin.

Although I've the feeling of it could be simpler, if it would have implemented like a simple ID for range, and then let the programmer, related it to any object he wanted.
Title: Re: Folding in SynEdit
Post by: Morton on October 13, 2013, 05:50:11 pm
I do not know who choose the name "range".

Maybe I should have choosen "scope" intead.
I introduced it in mwEdit to allow processing of syntax elements spanning over more than one line.
I was considering to implement it as a sub highlighter to make it more versatile,
but was to lazy to implement something so obvious (but not so simple to implement)
in something just meant to be an example.

Actually it didn't take long until one recognized the ability to have more than one highlighter in one editor,
but nobody did the obvious implementation in the base class.

It also holds the key to possible implementations of truly scriptable highlighters uncluding
folding and things like offered for delphi by Castalia.
Title: Re: Folding in SynEdit
Post by: Martin_fr on October 13, 2013, 06:23:35 pm
Hi Morton, great to have you here.
Always interesting to hear from these who started it all.

Also interesting idea of using a range, as a representation of multiple lines (or probably even parts of lines).

In SynEdit range has mutated to LineState: the state of the parser at the end of a line.


What would be the obvious think for a multi-highlighter in the base class? (sorry, but it may not be that obvious / or not anymore)

I can see (and not to hard) how to do a freely configurable hl with folding. But I do not see how ranges have much to do with that.

They are however useful for the folding. And Lazarus also uses them to find matching begin/end
Title: Re: Folding in SynEdit
Post by: Morton on October 13, 2013, 07:29:00 pm
In SynEdit range has mutated to LineState: the state of the parser at the end of a line.

I was just lazy.
It was the simplest way to imlement processing of multiline syntax elements.

Quote
What would be the obvious think for a multi-highlighter in the base class? (sorry, but it may not be that obvious / or not anymore)

Borland Pascal had allowed the use of two completely different languages within one text (Pascal and Assembler)
for a long time.
Multible highlighters are a very obvious solution to properly handle this in a highlighting editor.

Once implemented multible highlighters are very useful to handle other editing tasks,
especialy if the editor control canvas, the painter and the data buffer are seperated too.

Pascal (most modern languages too) can be easily broken into different language elements "ranges"
Basicaly all foldable language elements can be considered as ranges.

Handling all ranges by own highlighting objects looks very obvious for me.
Eg. Graphics as comments would be possible.
Title: Re: Folding in SynEdit
Post by: Martin_fr on October 13, 2013, 08:14:42 pm
Code: [Select]
Pascal (most modern languages too) can be easily broken into different language elements "ranges"
Basicaly all foldable language elements can be considered as ranges.

AHHH Yes..

That explains the word range.

Within a range (e.g. the body of a procedure versus the declaration of a class or interface vs implementation) there are different set of rules that apply.

So Storing the range, at the end of the line, allows to parse the next line in the context of that range.

--------------------

Quote
Once implemented multiple highlighters are very useful

Well there is an implementation of a multi hl. But it has quite some overhead.
1) It must decide where to break before handing down to the other hl, since the embedded HL do not know that they work on part of the text only.
E.g if pascal code is embedded into xml, the pascal HL would not know to stop at </code>
2) for the same reason it must present the other HL an extract of the complete text only

This could be optimized for a situation like asm in pascal. Both HL could know about the switching keywords. Therefore the overhead of the controller could be smaller.

Also the asm, can act on each asm block, without knowing the previous block.
In the pascal in xml, the source may be spread over several blocks, but must appear as continuous for the pas highlighter.

So the generic approach is very complex, but it does ekist: based on feeding the embedded HL with virtual documents.

Quote
to handle other editing tasks,
such as?
Something like viewing an include file embedded in the file that uses it? But that dose not need to combine the highlighters. So it is different.

Quote
especialy if the editor control canvas, the painter and the data buffer are seperated too.

that is work in progress.

I also added a distinction between highlight (based on the language) and Markup (based on language independent criteria e.g. block selection)

Title: Re: Folding in SynEdit
Post by: Morton on October 14, 2013, 02:58:56 pm
Well there is an implementation of a multi hl. But it has quite some overhead.

It is not generalized enough.

Just a hint for those who desire a scriptable beast capable to perform any lexical and syntactical generic expression.
Breaking highlighters (lexers in general) down to objects for small language elements
allows high generalisation and finaly to implement them as scriptable state machines.
State machines can perform very complex tasks, while the individual parts do not need to know
a lot about each others.

Disadvantages of scriptable machines:
Will never be able to compete with hard coded ones.
With increased power writting scripts that can take advantage of all features may become a very complex task, mastered by only a few.
Title: Re: Folding in SynEdit
Post by: Edson on October 19, 2013, 04:20:38 am
Quote
Breaking highlighters (lexers in general) down to objects for small language elements

Do you refer to the token or some deeper element?

I have found that for describing some tokens, like strings, it's necessary to use the concept of "delimiters", so they allow to define several kinds of tokens. On the other hand, we can use delimiters for defining too, ranges of multiples tokens.

So, for me, the delimiters are more basic elements, even than tokens. At least, that's how I have focused my work with highlighters.
Title: Re: Folding in SynEdit
Post by: Morton on October 19, 2013, 08:12:12 am
Do you refer to the token or some deeper element?

Anything that can be described with a generic expression.

Attached is an example implementation of a scriptable lexer state machine.
It is fast on Delphi and can be implemented nearly as fast on NET,
unfortunatly not as fast with FPC, but also not slow.
The state machines can be nested without limits.
Title: Re: Folding in SynEdit
Post by: Morton on October 19, 2013, 09:54:10 am
There had been an AnyHighlighter by James Jacobson using a previous version
of the state machine.
The AnyHighlighter in Lazarus has nothing to do with it.

If someone creates a state class for every imageable generic expression the state machine could handle everything imaginable.
Complete Compilers can be implemented as state machines.
Indeed everything that can be broken down into states.

A scriptable folding highlighter for any imageable language would be no problem.
It just has to done, but not by me.
Title: Re: Folding in SynEdit
Post by: Edson on January 24, 2014, 05:06:28 pm
Here I'm again with folding.

I was analyzing the code of folding on the SynLFMSyn and some others HL, and my question is:

What's the difference on implementing folding, using:

Code: [Select]
function TSynLFMSyn.StartLfmCodeFoldBlock(ABlockType: TLfmCodeFoldBlockType): TSynCustomCodeFoldBlock;
var  FoldBlock: Boolean;
  p: PtrInt;
begin
  FoldBlock :=  FFoldConfig[ord(ABlockType)].Enabled;
  p := 0;
  if not FoldBlock then
    p := PtrInt(CountLfmCodeFoldBlockOffset);
  Result := StartCodeFoldBlock(p + Pointer(PtrInt(ABlockType)), FoldBlock);
end;

procedure TSynLFMSyn.EndLfmCodeFoldBlock;
var  DecreaseLevel: Boolean;
begin
  DecreaseLevel := TopCodeFoldBlockType < CountLfmCodeFoldBlockOffset;
  EndCodeFoldBlock(DecreaseLevel);
end;

And simply:

Code: [Select]
    StartCodeFoldBlock(nil);

    EndCodeFoldBlock();

I know it has to be with the manage of ranges of TLfmCodeFoldBlockType. But how it works?
There is not documentation about StartCodeFoldBlock().
Title: Re: Folding in SynEdit
Post by: Martin_fr on January 24, 2014, 08:24:58 pm
I added some info

http://wiki.lazarus.freepascal.org/SynEdit_Highlighter#Step_3:_Add_Folding

The pascal variants:
- typecast the
Code: [Select]
  TPascalCodeFoldBlockType = ( // Do *not* change the order
    cfbtBeginEnd,      // Nested
    cfbtTopBeginEnd,
into a pointer
- encode (by adding an offset number), if a fold starts here.
  In the EndFoldBlock is a check if "TopCodeFoldBlockType" is greater/equal to the offset.

FCatchNodeInfo should be ignored for now. It will neither be documented, nor will it be kept compatible, until it moves to the base class.
Title: Re: Folding in SynEdit
Post by: Edson on January 24, 2014, 10:21:02 pm
Thaks Martin.

It's good to have more documentation on the Wiki  :). It clears some of my doubts.

Would you please tell me if the parameter 'ABlockType' (a pointer)  have the same use that the parameter 'Value', when we use Ranges:

Code: [Select]
procedure SetRange(Value: Pointer); virtual;
 function GetRange: Pointer; virtual;
 procedure ResetRange; virtual;

I mean,
It's stored internally, for each line of the text?
Is it read at the end of each line?.
Can I use it for storage (point to) some complex object?
Title: Re: Folding in SynEdit
Post by: Martin_fr on January 24, 2014, 10:40:53 pm
No they are different.

Look at the doc, if you use the fold-hl, then the RANGE is managed by the base class.
It becomes an object(you can change the class of it, if need it, as long as it has the right base class)

Your range, is then in
 FCurRange := PtrInt(CodeFoldRange.RangeType);

The ID only identifies the fold type e.g. in pascal
begin=1
try=2
repeat=3
...

The range PtrInt(CodeFoldRange.RangeType) can still be used for other things. Thinks where you do not want a foldblock.

Foldblocks have the advantage that
- they can be nested
- you can find the start line very fast (see the section on "EndLvl")

Yet if you have range_InAssignment, (that is between ":=" and ";") then you need no foldblock.
You can use the range as enum/set, and flag the state there.


Title: Re: Folding in SynEdit
Post by: Edson on January 25, 2014, 02:35:10 am
Understood.

I have enough information for continuing working.

In my case, I can't use enumerated ID's for the fold type, because I need to create them dynamically, and store different properties on them.

I'm going to use real pointers for objects on ABlockType of StartCodeFoldBlock().
Title: Re: Folding in SynEdit
Post by: Martin_fr on January 25, 2014, 03:27:42 am
I am not sure what you want to store, but it does not sound ... well ...

look also at

    function GetRangeClass: TSynCustomHighlighterRangeClass; override;

unit
SynHighlighterPas

It returns TSynPasSynRange

For the pas HL, on each line an object of this class is stored.

IMPORTANT: it must have a compare method that compares *ALL* (each and every) field (or piece of data) on the class (see TSynPasSynRange.Compare). Inherited data is done in inherited.
(also assign)

PascalHL has

from inherited
  property RangeType: Pointer read FRangeType write FRangeType;
which is a set of enum (TRangeState)

folds for begin/end etc

   property BracketNestLevel: integer read FBracketNestLevel write FBracketNestLevel;
a counter for ()

counter for IFDEF folds

....

-----------------
Instead of subclassing, you can also store an object in
  property RangeType: Pointer read FRangeType write FRangeType;

But subclassing is more efficient. It will avoid storing duplicate objects.



Title: Re: Folding in SynEdit
Post by: Edson on January 25, 2014, 06:03:16 pm
This leaves me more questions. :o

Quote
    function GetRangeClass: TSynCustomHighlighterRangeClass; override;

What is this for?

Quote
For the pas HL, on each line an object of this class is stored.

Why is needed to store an object per line?

Quote
PascalHL has

from inherited
  property RangeType: Pointer read FRangeType write FRangeType;
which is a set of enum (TRangeState)

folds for begin/end etc

For me, "RangeType" is used for manage the "ranges" for highlight multiline tokens (or other information per line). Isn't it?

I'm getting some confused.

For all I know:

Ranges
=====================================
When we use a HL derived from "TSynCustomHighlighter", we only manage "Ranges", for store some information per line and/or whose state depend on previous line.
We do this, using an enumerated "fRange" and implementing:

Code: [Select]
procedure TSynMiColor.ReSetRange;
begin
  fRange := rsUnknown;
end;

function TSynMiColor.GetRange: Pointer;
begin
  Result := Pointer(PtrInt(fRange));
end;

procedure TSynMiColor.SetRange(Value: Pointer);
begin
  fRange := TRangeState(PtrUInt(Value));
end;


Blocks of Folding
=====================================
When we use a HL derived from "TSynCustomFoldHighlighter", we manage the "Ranges", and aditionally "Blocks of folding" (do we call them Ranges too?) for store information about the folding (included nesting).
We use too an enumerated "fRange" (for common ranges) and we have to implement:

Code: [Select]
procedure TSynFacilSyn.ReSetRange;
begin
  inherited;
  fRange := nil;
end;
function TSynFacilSyn.GetRange: Pointer;
begin
  CodeFoldRange.RangeType := fRange;
  Result := inherited GetRange;
end;
procedure TSynFacilSyn.SetRange(Value: Pointer);
begin
  inherited SetRange(Value);
  fRange := CodeFoldRange.RangeType;
end;

Is it OK? Probably I'm wrong in my concepts or terms.

I think they are different things. But, are they (ranges and blocks) related in some way?
Title: Re: Folding in SynEdit
Post by: Martin_fr on January 25, 2014, 06:57:11 pm
Ranges:

You may remember from some other thread, that apparently ranges were initially a "range of lines to which applies some condition".
For each line we store that "Range".

The range is stored on each line. If it is not changed, then the same range is stored again.


Quote
For me, "RangeType" is used for manage the "ranges" for highlight multiline tokens (or other information per line). Isn't it?

In some Highlighters this ranges are now very detailed.
Code: [Select]
type
  TFoo = class
  // ..
  end  ; 
  deprecated
Will NOT highlight the word "deprecated", but if you take the ";" away, then it will. So after the ";" starts a new range, or the other way round, there was a range that ended at the ";".

Being in a fold able block, is also part of the range. It may not (always) be something that affects how the code in this lines is parsed, but it is an attribute that applies to those lines (and besides it is a convenient place to store it)

So anything the HL needs to store, that is needed in a later line, goes into the range.

e.g. fold info is needed at the closing line. It is carried forward. So no loo back is needed.


Quote
Why is needed to store an object per line?
Because the amount of flags and info does not fit into the 32 or 64 bits of a pointer.

But yes, it is also true to extend the storage size in the array of ranges (1 per line), so there are 128 bits.

---------------
As before: The range is stored on each line. If it is not changed, then the same range is stored again.

If it is an object, ideally we do not copy the object, but duplicate the reference. However then the object stored must be immutable. If you changed it, that would affect many lines instead of just one.

The fold base HL does introduce all that is needed to deal with such an object. And it forces all inherited HL do use that object.

Quote
and aditionally "Blocks of folding" (do we call them Ranges too?)
The entire object is called the "range" now. (Actually the range is the part of code/text to which it applies).

The tutorial only showed the basics.

The range can hold whatever is needed.
But if designed well, you must look at implications such as memory.

Look at pascal, using objects. Open the IDE, and open a handful of units. You easily have 100.000 lines or more of source. Imagine a range object for each line. that are a lot of objects.
In fact, even with 1 million lines, the IDE only need approx 2000 objects In rare cases I have seen 4000.

-----------------------------
If you use the fold ID to point to an object, then you need to manage this your same.
You either get one per line, or must organize to re-use them. Well you get one for each fold-opening line, but worst case that is every line)

BTW: Those ID, are stored as part of the range-object.
Title: Re: Folding in SynEdit
Post by: Martin_fr on January 25, 2014, 07:11:23 pm
Also note, that ranges (and changing of their value) is used to find out how far the HL must scan.

Code: [Select]
if a then begin
  (* // Some commented initialization:
  x := 0;
  b := a;
  Init(Foo);
  // *)
  Foo.DoSome;
end;

Now if the line
Code: [Select]
  b := a;is edited, and changed into
Code: [Select]
  b := a; *)then only the text in that line changed.

That means to start with, only that line needs to be scanned. But of course that is not enough.
So a way is needed to know how far the scan must go. After all, we also do not want to scan to the very end of the file each time (that could be a million lines)

So when
Code: [Select]
  b := a; *)it will look at the range that it returns for the end of that line.
It compares it to the old value.

* If it is equal it is done.
* If it is changed, then the next line now start in a different range, and the next line must be scanned too,

And this is repeated, until a range is kept equal.
In the example, that is at the end of " Foo.DoSome;"
Title: Re: Folding in SynEdit
Post by: Edson on January 25, 2014, 08:36:16 pm
Thanks Martin.

I think I understand the concept of Range (like pointers stored on each line). I have implemented some hardcoded Highlighters, and in my Scriptable Highlighter I use the range like pointer to a record and it works OK (maybe I'm the only tester).

But what confuse me, is the terminology on folding. I see "ranges" and "blocks", and I have no definitions.
I can suppose that block is "A logical representation of some extension of text that can be folded and nested". Is it OK?

Probably for programming, the concepts are not important (we create our own in mind), but for documentation yes.

In some Highlighters this ranges are now very detailed.
Code: [Select]
type
  TFoo = class
  // ..
  end  ; 
  deprecated
Will NOT highlight the word "deprecated", but if you take the ";" away, then it will. So after the ";" starts a new range, or the other way round, there was a range that ended at the ";".

I guess you are talking about that, HL can use the information of Folding for parsing. If so, then it's clear.

But I still have the same my question:

Are Ranges and Blocks(of folding) different things?

I mean, that we can use one of then without implementing (or leave for default) the other?

I think the response is Yes. But it will be nice have your opinion.

Title: Re: Folding in SynEdit
Post by: Martin_fr on January 25, 2014, 09:10:12 pm
The terminology is used quite loosely. After all the original meaning of "range" was not known any more until recent (thanks to morton for explaining it).

"Range" is still used in many ways. Does identify a "code snippet"? Or some ID (the sum of the data stored in the field called range)?

If we go by "a range of lines" then I would say the actual code....
And it can start in the middle of a line. It can even end on the same line it started.
The info stored in the range field, is just a snapshots of the identification of the range (a snapshot at the end of each line).

That is why I like to think of it as "the internal state of the Highlighter at the end of a line"

----
Block (if used as foldblock) then yes, some form of reference to foldable code.

In that sense a block is part of the information stored about a range.

Quote
Are Ranges and Blocks(of folding) different things?
A fold block is one way to start a new range.

fold-block-info is part of the range info.

Quote
I guess you are talking about that, HL can use the information of Folding for parsing. If so, then it's clear.

It can. It does not always have too. It sometimes does not need it at all.

I was not trying to make any statement about this.

Quote
I mean, that we can use one of then without implementing (or leave for default) the other?

You can use ranges, without folding, every none fold HL does that.

The foldHLbase implements folds-info as part of the range. So it always uses both.

If you open/close a fold, then the base class changes the current range. (as it stores the fold info within)


You can of course use folds, and not add any of your own info to that range. (but the range info from the base class is still there.





Title: Re: Folding in SynEdit
Post by: Edson on January 25, 2014, 10:57:03 pm
It's clearer now.

Although the Range and Fold Block, have different uses (but could be related), they both use the same mechanism of ranges for to work. It has sense.

Quote
A fold block is one way to start a new range.

fold-block-info is part of the range info.

If the fold block info and the range info are shared. Is there some limitation on that.?
I say it, because I'm using pointer for ranges and for fold blocks (not enumerated ID's), so I'm using the 32 bits of data. I haven't noticed some problem by now.

Until now, I have been success with scritptable foldable HL. Just I'm having problems with nested foldable blocks. I have the next doubts:

A Start delimitier (like "BEGIN"), can only open a block. Is it OK?

But an End delimitier (like "END"), can close two fold blocks: PROCEDURE ... END and BEGIN ... END

Can it close 3 or more blocks?
Title: Re: Folding in SynEdit
Post by: Martin_fr on January 25, 2014, 11:34:04 pm
It is the highlighters decision how many blocks to open or close.

Say I wanted

try
except
end

to be able to fold either "try except" or "try end" then I would open 2 blocks at the try.

But there  can only be one [-] in the gutter, so it is very hard to access (gutter pop up menu)

Overall in pascal it makes no sense to open more than one block for a keyword.

-----
"end"
look at
Code: [Select]
function TSynPasSyn.Func23: TtkTokenKind;
  if KeyComp('End') then begin
It can close a lot.

Code: [Select]
Case a of
  1: foo
   else bar
end;
NOTE: no ";" before the case-else needed.

Code: [Select]
Case a of
  1: If x1 then
        If x1 then
           If x1 then foo
          //else bar
end;
     
Each "then" opens an internal block (it is marked not to be foldable / does NOT increase the fold-level), because it means the "else" belongs to the IF, and not to the case.

So above could have 3 else for the 3 if, and one for the case.

The end closes them all, and it closes the case.
Title: Re: Folding in SynEdit
Post by: Edson on January 26, 2014, 12:10:19 am
Thanks Martin.

I can see. There is several figures, not all present on Pascal language.

Now I just have to find the way to express it on a XML description.

By now, I have:

<block> %) </block>

Title: Re: Folding in SynEdit
Post by: Edson on April 07, 2014, 02:17:03 am
Hi Martin,

I was trying to get the information of ranges from a Highlighter, for any line (specifically the current line).

I read a pointer, and then I apply the Typecast:

Code: [Select]
var
  hlr: TSynCustomHighlighterRange;
  p: Pointer;
  ...
  p := CurrentRanges[SynEdit1.CaretY-1]
  hlr := TSynCustomHighlighterRange(p);

I think this is OK until now.

I can read the RangeType (at the end of the line) like pointer in: hlr.RangeType

But, how can I get the Fold Block information?

I see the properties: 'hlr.MinimumCodeFoldBlockLevel' and 'hlr.CodeFoldStackSize', that I guess, are the mentioned here:
http://wiki.lazarus.freepascal.org/SynEdit_Highlighter#Folding

But I don't know how to obtain the 'ABlockType' used with StartCodeFoldBlock(). Is this captured at the end of the line, too? And the nested levels?

Probably I'm on the wrong way. My objective is to obtain all the Fold information of the current Caret position.

Thanks.
Title: Re: Folding in SynEdit
Post by: Martin_fr on April 07, 2014, 03:16:51 am
It depends what fold info you exactly need.....


1) Yes your code sample is right. That is, if the HL inherit from TSynCustomFoldHighlighter.
Other HL may have data of different type in the Range. But TSynCustomFoldHighlighter does TSynCustomHighlighterRange

2)
Look at
function TSynCustomFoldHighlighter.TopCodeFoldBlockType(DownIndex: Integer = 0): Pointer;

It shows how to get "ABlockType"


However:
Code: [Select]
Procedure a;               1 -      0
 Begin                      2 --     1 -
   b:= 1;                   2 --     2 --
   if c > b then begin      3 ---    2 --
     c:=b;                  3 ---    3 ---   // <<<<< If you query this line
   end else begin           3 ---    2 --
     b:=c;                  3 ---    3 ---
   end;                     2 --     2 --
 end;                       0        0 

You will be able to know:
- there are 3 folds open
- they have the "ABlockType" cfbtProcedure, cfbtTobBegin, cfbtBegin
 Or whatever the blocktypes are in your code.

You can find the line at which they start/end, by looking at MinLevel. That is explained on the wiki. Or look at
function TSynCustomFoldHighlighter.FoldLineLength(ALineIndex, FoldIndex: Integer): integer;


You can *NOT* find out, at which column the fold started/ended.

Well I know it is done for Highlighting begin/end pairs. But it is not documented, and probably will change, as it needs clean up.
Also this is not done via the ranges. The pas highlighter supports a special way of scanning a single line. In this scan it will record the column info.
So to find anything within a line, special scans are needed.
(search for "FCatchNodeInfo", but as I said, It is intentionally not documented)
Title: Re: Folding in SynEdit
Post by: Edson on April 07, 2014, 06:08:17 am
I need the Start and the End of the Fold Block, where the Cursor is. I need this information for to highlight the Current Block.

I think TSynCustomFoldHighlighter.TopCodeFoldBlockType() is not working for me, because it is for Read the Top Fold at the current time, when it is scanned.

I need to read information about block in ANY time. It's, every time the cursor is moved, or some text is typed. I'm trying to avoid re-scan the text wtih the HL, every time the cursor is moved.

TSynCustomFoldHighlighter.FoldLineLength()? How does this function work?

I have found that:

TSynCustomHighlighterRange.Top.BlockType

gives the Fold Block, at the end of the line. Is't true?

It could be useful.

Title: Re: Folding in SynEdit
Post by: Martin_fr on April 07, 2014, 03:39:42 pm
What stops you from getting the range of any line you want?

Look at TSynCustomFoldHighlighter.FoldBlockEndLevel (It is designed to be called *outside* scanning). It gets the FoldEndLevel for any line.

However, it assumes that all scanning is done. So if you modify the textbuffer, you must first call scan, then you can call this. (SynEdit does this in the last EndUpdate)

You can write yourself a method that gets the blocktype in that way.

---
There also is TLazSynEditNestedFoldsList which is a helper for all this.

----
TSynCustomFoldHighlighter.FoldLineLength()

It just accesses the Ranges for all the lines it want. Once scanned the ranges can be accessed at random. they are stored in a big list.

----

TSynCustomHighlighterRange( CurrentRanges[SynEdit1.CaretY-1] ) .Top.BlockType

that is the type of the last fold opened, before the end of the indicated line. That fo[d may have opened on that line, or it may have opened on a previous line.

- If MinLevel < EndLevel, then it opened on this line.
- If MinLevel = EndLevel, then it opened on a previous line

----
You should always cache the current begin/end pair.
If you are on line 200, and the nearest block starts at 190, and goes to 225, then you can cache his. If the cursor moves without editing, then the info is still valid.


----
SynEdit1.CaretY-1

Hint:

For readability use ToIdx(SynEdit1.CaretY)

Then it is clear you talk about the caret line, not the line befor the caret

Idx (Index) = 0 based
Pos (Position) = 1 based

and there are
  ToIdx(ALinePos)
  ToPos(ALineIndex)
Title: Re: Folding in SynEdit
Post by: Martin_fr on April 07, 2014, 04:02:19 pm
Almost forgot. Before any access to the highlighter
Code: [Select]
Highlighter.CurrentLines := SynEdit.FLines; //

Otherwise the HL does not know, which SynEdit...
Title: Re: Folding in SynEdit
Post by: Edson on April 07, 2014, 10:29:09 pm
OK, I can obtain the MinLevel and the EndLevel for each line, using:

TSynCustomFoldHighlighter.FoldBlockMinLevel()
TSynCustomFoldHighlighter.FoldBlockEndLevel()

And I can obtain the last BlockType opened, at the end of the line using:

TSynCustomHighlighterRange( CurrentRanges[SynEdit1.CaretY-1] ) .Top.BlockType (Is there a short form?)

But, How can I can obtain the other BlockType's (of the nested blocks) for every line? Is it saved in some place?

There also is TLazSynEditNestedFoldsList which is a helper for all this.

How can I use this class?
Title: Re: Folding in SynEdit
Post by: Martin_fr on April 07, 2014, 10:42:07 pm
TSynCustomHighlighterRange( CurrentRanges[SynEdit1.CaretY-1] ) .Top.BlockType (Is there a short form?)

But, How can I can obtain the other BlockType's (of the nested blocks) for every line? Is it saved in some place?
Look at
function TSynCustomFoldHighlighter.TopCodeFoldBlockType(DownIndex: Integer = 0): Pointer;

This is for the range during the scan. But if ou get a range outside the scan you can do the same

Quote

There also is TLazSynEditNestedFoldsList which is a helper for all this.

How can I use this class?

Look at \ide\sourcesyneditor.pas
TIDESynEditor.SrcSynCaretChanged(

      List := TextView.FoldProvider.NestedFoldsList;
      List.ResetFilter;
      List.Clear;
      List.Line := CaretY-1;
      List.FoldGroup := FOLDGROUP_PASCAL; // laways 0 (zero)
      List.FoldFlags := [sfbIncludeDisabled];  // only matters, if your HL has such config
      List.IncludeOpeningOnLine := False;

List.NodeLine[0]  // line on which the most inner fold starts
List.NodeLine[1]  // line on which the ONE outside the most inner fold starts

List.NodeFoldType  the type


Title: Re: Folding in SynEdit
Post by: Edson on April 08, 2014, 10:57:06 pm
Thanks Martin.

Finally, I have used TSynCustomHighlighter.StartAtLineIndex(), for set the position of the HL, then I explore the current line until raise the CaretX. Then I use TopCodeFoldBlockType() for get the Current Top Folding Block.

It seems to work by now.

I have some bug on detecting the Top block at the end of some blocks, but it must be problem of the block's manage of my HL.

PS: It should be very good if the Source Code of the IDE would have comments.
Title: Re: Folding in SynEdit
Post by: Edson on April 12, 2014, 09:06:54 pm
I have success on having the Current Fold Block. But I'm having problems obtaining the limits of that block. I manage several blocks on the same line. That's why I need to read the column information.

Some questions:

1. What's the meaning of the parameter "FoldIndex" on the function TSynCustomFoldHighlighter.FoldLineLength() ?
2. What's FoldEndLine() for?. I guess this gives the end line, of the last block opened at the end of the indicated Line. But I find some errors on his work.
3. Is there some direct way for to obtain the quantity of ALL levels of folding? Accessing to TSynCustomFoldHighlighter.CodeFoldRange.CodeFoldStackSize, give me only the blocks opened with IncreaseLevel=TRUE.

Thanks.
Title: Re: Folding in SynEdit
Post by: Martin_fr on April 12, 2014, 09:44:22 pm
1. What's the meaning of the parameter "FoldIndex" on the function TSynCustomFoldHighlighter.FoldLineLength() ?
IIRC (please test) the index of the fold. That is, if there are more than one fold opening in that line (e.g. in line 20) then
  FoldLineLength(20,0) returns the amount of lines to the end of the first fold (leftmost) opening on that line
  FoldLineLength(20,1)  returns for the 2nd fold opening in that line.

This only accounts for folds that do NOT close on the same line. Folds closing on the same line are ignored, as they can not be folded.

Quote
2. What's FoldEndLine() for?. I guess this gives the end line, of the last block opened at the end of the indicated Line. But I find some errors on his work.

Returns the line number (or index,  not sure 0 versus 1 based) of the line on which the closing token of the fold is.

It does so by using end- and min-levels.
Again it ignores anything that closes on the same line as it opens.

Quote
3. Is there some direct way for to obtain the quantity of ALL levels of folding? Accessing to TSynCustomFoldHighlighter.CodeFoldRange.CodeFoldStackSize, give me only the blocks opened with IncreaseLevel=TRUE.

Not sure what you mean?

If you mean that you open/close some folds with In-/De-creaseLevel, and you want
- the total of al that had In-/De-creaseLevel
- AND the total of all, including the ones without In-/De-creaseLevel

Then look at  TSynPasSynRange
It adds the 2nd counter.
Also search for sfbIncludeDisabled (must be implemented by the highlighter)



Quote
I manage several blocks on the same line. That's why I need to read the column information.

Then you need to scan the line, when you need the info, and capture the info.
Pas HL does that. But it needs a lot of clean up, and the structures involved in that may change in future (Well maybe / maybe not so much...). But it should be in such way, that your sources can then be updated too. (I do recommend basic unit testing on that stuff)

Search for FCatchNodeInfo in the pas hl.

The info can be queried using       
List := TextView.FoldProvider.NestedFoldsList; (see  foldactions and stuff)

Unfortunately I havent worked on it in a while, so you need to search the sources.
Title: Re: Folding in SynEdit
Post by: Edson on April 14, 2014, 01:32:20 am
Thanks Martin for the responses.

Yes. "FoldIndex" is the Nesting Level of Folding. I haven't tested at all, but it seems to be that.

I think that Folding in TSynCustomFoldHighlighter, have been designed just for support the Folding marks on the gutter. It's oriented to lines.

The MinLevel and EndLevel, just consider the visible blocks. That's not enough for my HL.

I see Pascal HL, use another counters. But it's difficult for me to follow undocumented source. I'm not good on reading other's mind.

Anyway I need to scan lines, and need some extra counters. On the other hand, I don't want to loose speed on extra processing

I'm going to check and document some methods of the unit SynEditHighlighterFoldBase. It will take me some time.
Title: Re: Folding in SynEdit
Post by: Martin_fr on April 14, 2014, 02:12:45 am
With maybe 50 or 100 lines visible, that may need extra scanning, speed is not an issue.
Storing the extra info for 10000 lines, would probably be more of an issue.

Also, you can store a flag, if there is anything or nothing on that line. Then it will be even less of an effort.

The pas HL does not even do that flag, and yet it is super fast.

It only needs the extra scan for very few lines, such as the line at caret., So doing an extra scan does not matter. If it needs more, a flag may be added.

---
Adding the extra counter for with/without In-/De-creaseLevel should be easy.
I can get you more on that, if you need.

---
But the column finding code is another matter. It is a lot of code, and to document, or explain from scratch would take a lot of time.

Just basic notes:
1) During normal scanning, columns are not recorded.
2) When needed a flag (FCatchNodeInfo ) is set, and then a single line will be scanned again (or several)
3) When the flag is set, for each keyword information is added to a list.

You can see where the flag is tested "if FCatchNodeInfo " in open/close-fold procedures.
Ignore that there are different procedures for open/close, that has nothing to do with FCatchNodeInfo . Just look at one open and one close-fold method.

You can see the record where the info is stored, as well the list to which it is added.

The rest, sorry, is up to you.
TinyPortal © 2005-2018