Recent

Author Topic: Folding in SynEdit  (Read 27171 times)

Edson

  • Hero Member
  • *****
  • Posts: 1086
Folding in SynEdit
« on: August 07, 2013, 11:23:44 pm »
Hi,

I was implementing the "Folding" on a simple Editor, and reviewing the code of TSynLFMSyn, i noted the use of "CodeFoldRange" property. And I have some questions:

* What is "CodeFoldRange" for?
* Is there some kind of conflict on using ranges for Folding and ranges for Context Highlighter?

Thanks on advance.

PD: I'm using Lazarus 1.0.8 on i386/Win32.
Lazarus 2.0.6 - FPC 3.0.4 - x86_64-win64 on  Windows 8

Tristan32

  • Newbie
  • Posts: 3
Re: Folding in SynEdit
« Reply #1 on: August 07, 2013, 11:34:08 pm »
Don't know if this will help, but found this in here - https://github.com/alrieckert/lazarus/blob/master/components/synedit/synedithighlighterfoldbase.pas Line 768

Code: [Select]
// FCodeFoldRange is the working range and changed steadily
  // => return a fixed copy of the current CodeFoldRange instance,
  // that can be stored by other classes (e.g. TSynEdit)

Martin_fr

  • Administrator
  • Hero Member
  • *
  • Posts: 6611
  • Debugger - SynEdit - and more
    • wiki
Re: Folding in SynEdit
« Reply #2 on: August 08, 2013, 09:43:58 am »
the codefoldrange is a subclass of the normal range.

"range" represents the state of the scanner. the state is stored at the end of each line, so it can be used to continue on the next line.

"range" objects are immutable. the same object is used for all lines that end with the same state. (if the state changes (line edited), then another object is used, because the existing one must not be changed).

codefoldrange stores the nesting depth and type of fold. it also offers storage for the plain "range" info.

not sure if you can avoid it, since you are expected to inherit the highlighter from  TSynCustomFoldHighlighter.

http://wiki.lazarus.freepascal.org/SynEdit_Highlighter

----
Edit:

a range can be mutated if it is not stored yet.

so a working copy may be mutable
« Last Edit: August 08, 2013, 09:53:33 am by Martin_fr »

Edson

  • Hero Member
  • *****
  • Posts: 1086
Re: Folding in SynEdit
« Reply #3 on: August 10, 2013, 12:11:05 am »
Thanks for the responses.

I've implemented a simple folding, and it works. But I want to understand the philosophy of the folding, because I'm preparing a documentation.

Martin, when you say "... the normal range", Are you refering to the "TSynCustomHighlighterRange" class?.

Are the functions GetRange(), SetRange() expecting to work with a pointer to "TSynCustomHighlighterRange"?

I feel a little confused.


Thanks.
Lazarus 2.0.6 - FPC 3.0.4 - x86_64-win64 on  Windows 8

Martin_fr

  • Administrator
  • Hero Member
  • *
  • Posts: 6611
  • Debugger - SynEdit - and more
    • wiki
Re: Folding in SynEdit
« Reply #4 on: August 10, 2013, 08:48:01 am »
I do not know who choose the name "range". probably that was done in the original SynEdit, before it was forked for lazarus.

"range" can be an integer or enum/set (cast to a pointer), or an object (does not have to be TSynCustomHighlighterRange, but I haven't seen any other class being used)

see also the link from my last post.
http://wiki.lazarus.freepascal.org/SynEdit_Highlighter#Step_2:_Using_Ranges
uses an integer.

Edson

  • Hero Member
  • *****
  • Posts: 1086
Re: Folding in SynEdit
« Reply #5 on: August 14, 2013, 02:17:36 am »
I'm understanding.

Someone designed the manage of ranges through pointers, in order to have the ability for using objects (although we prefer to use enumerated, integers or sets).

But, In which cases it's useful to use objects as ranges?

Thanks.
Lazarus 2.0.6 - FPC 3.0.4 - x86_64-win64 on  Windows 8

Martin_fr

  • Administrator
  • Hero Member
  • *
  • Posts: 6611
  • Debugger - SynEdit - and more
    • wiki
Re: Folding in SynEdit
« Reply #6 on: August 14, 2013, 07:55:15 am »
But, In which cases it's useful to use objects as ranges?

1) when the data doesn't fit into 32 bit (assuming your app should run on 32 bit cpu too)

2) folding, because the baseclass does.

Edson

  • Hero Member
  • *****
  • Posts: 1086
Re: Folding in SynEdit
« Reply #7 on: August 14, 2013, 06:33:45 pm »
It's clear. Thanks Martin.

Although I've the feeling of it could be simpler, if it would have implemented like a simple ID for range, and then let the programmer, related it to any object he wanted.
Lazarus 2.0.6 - FPC 3.0.4 - x86_64-win64 on  Windows 8

Morton

  • Full Member
  • ***
  • Posts: 111
Re: Folding in SynEdit
« Reply #8 on: October 13, 2013, 05:50:11 pm »
I do not know who choose the name "range".

Maybe I should have choosen "scope" intead.
I introduced it in mwEdit to allow processing of syntax elements spanning over more than one line.
I was considering to implement it as a sub highlighter to make it more versatile,
but was to lazy to implement something so obvious (but not so simple to implement)
in something just meant to be an example.

Actually it didn't take long until one recognized the ability to have more than one highlighter in one editor,
but nobody did the obvious implementation in the base class.

It also holds the key to possible implementations of truly scriptable highlighters uncluding
folding and things like offered for delphi by Castalia.

Martin_fr

  • Administrator
  • Hero Member
  • *
  • Posts: 6611
  • Debugger - SynEdit - and more
    • wiki
Re: Folding in SynEdit
« Reply #9 on: October 13, 2013, 06:23:35 pm »
Hi Morton, great to have you here.
Always interesting to hear from these who started it all.

Also interesting idea of using a range, as a representation of multiple lines (or probably even parts of lines).

In SynEdit range has mutated to LineState: the state of the parser at the end of a line.


What would be the obvious think for a multi-highlighter in the base class? (sorry, but it may not be that obvious / or not anymore)

I can see (and not to hard) how to do a freely configurable hl with folding. But I do not see how ranges have much to do with that.

They are however useful for the folding. And Lazarus also uses them to find matching begin/end

Morton

  • Full Member
  • ***
  • Posts: 111
Re: Folding in SynEdit
« Reply #10 on: October 13, 2013, 07:29:00 pm »
In SynEdit range has mutated to LineState: the state of the parser at the end of a line.

I was just lazy.
It was the simplest way to imlement processing of multiline syntax elements.

Quote
What would be the obvious think for a multi-highlighter in the base class? (sorry, but it may not be that obvious / or not anymore)

Borland Pascal had allowed the use of two completely different languages within one text (Pascal and Assembler)
for a long time.
Multible highlighters are a very obvious solution to properly handle this in a highlighting editor.

Once implemented multible highlighters are very useful to handle other editing tasks,
especialy if the editor control canvas, the painter and the data buffer are seperated too.

Pascal (most modern languages too) can be easily broken into different language elements "ranges"
Basicaly all foldable language elements can be considered as ranges.

Handling all ranges by own highlighting objects looks very obvious for me.
Eg. Graphics as comments would be possible.

Martin_fr

  • Administrator
  • Hero Member
  • *
  • Posts: 6611
  • Debugger - SynEdit - and more
    • wiki
Re: Folding in SynEdit
« Reply #11 on: October 13, 2013, 08:14:42 pm »
Code: [Select]
Pascal (most modern languages too) can be easily broken into different language elements "ranges"
Basicaly all foldable language elements can be considered as ranges.

AHHH Yes..

That explains the word range.

Within a range (e.g. the body of a procedure versus the declaration of a class or interface vs implementation) there are different set of rules that apply.

So Storing the range, at the end of the line, allows to parse the next line in the context of that range.

--------------------

Quote
Once implemented multiple highlighters are very useful

Well there is an implementation of a multi hl. But it has quite some overhead.
1) It must decide where to break before handing down to the other hl, since the embedded HL do not know that they work on part of the text only.
E.g if pascal code is embedded into xml, the pascal HL would not know to stop at </code>
2) for the same reason it must present the other HL an extract of the complete text only

This could be optimized for a situation like asm in pascal. Both HL could know about the switching keywords. Therefore the overhead of the controller could be smaller.

Also the asm, can act on each asm block, without knowing the previous block.
In the pascal in xml, the source may be spread over several blocks, but must appear as continuous for the pas highlighter.

So the generic approach is very complex, but it does ekist: based on feeding the embedded HL with virtual documents.

Quote
to handle other editing tasks,
such as?
Something like viewing an include file embedded in the file that uses it? But that dose not need to combine the highlighters. So it is different.

Quote
especialy if the editor control canvas, the painter and the data buffer are seperated too.

that is work in progress.

I also added a distinction between highlight (based on the language) and Markup (based on language independent criteria e.g. block selection)


Morton

  • Full Member
  • ***
  • Posts: 111
Re: Folding in SynEdit
« Reply #12 on: October 14, 2013, 02:58:56 pm »
Well there is an implementation of a multi hl. But it has quite some overhead.

It is not generalized enough.

Just a hint for those who desire a scriptable beast capable to perform any lexical and syntactical generic expression.
Breaking highlighters (lexers in general) down to objects for small language elements
allows high generalisation and finaly to implement them as scriptable state machines.
State machines can perform very complex tasks, while the individual parts do not need to know
a lot about each others.

Disadvantages of scriptable machines:
Will never be able to compete with hard coded ones.
With increased power writting scripts that can take advantage of all features may become a very complex task, mastered by only a few.

Edson

  • Hero Member
  • *****
  • Posts: 1086
Re: Folding in SynEdit
« Reply #13 on: October 19, 2013, 04:20:38 am »
Quote
Breaking highlighters (lexers in general) down to objects for small language elements

Do you refer to the token or some deeper element?

I have found that for describing some tokens, like strings, it's necessary to use the concept of "delimiters", so they allow to define several kinds of tokens. On the other hand, we can use delimiters for defining too, ranges of multiples tokens.

So, for me, the delimiters are more basic elements, even than tokens. At least, that's how I have focused my work with highlighters.
Lazarus 2.0.6 - FPC 3.0.4 - x86_64-win64 on  Windows 8

Morton

  • Full Member
  • ***
  • Posts: 111
Re: Folding in SynEdit
« Reply #14 on: October 19, 2013, 08:12:12 am »
Do you refer to the token or some deeper element?

Anything that can be described with a generic expression.

Attached is an example implementation of a scriptable lexer state machine.
It is fast on Delphi and can be implemented nearly as fast on NET,
unfortunatly not as fast with FPC, but also not slow.
The state machines can be nested without limits.
« Last Edit: October 19, 2013, 09:10:03 am by Morton »

 

TinyPortal © 2005-2018