Lazarus

Programming => Packages and Libraries => SynEdit => Topic started by: A-M on October 22, 2015, 09:02:48 pm

Title: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: A-M on October 22, 2015, 09:02:48 pm
Hi Dear FPC/Lazarus Developers
The idea is so simple , colorizing code blocks start and ending keywords for helping on perception while developing and browsing the source code
like what the CNPack add-on do on delphi IDE

its specially important when you are developing a complex algorithm with multi level code blocks inside ...
and even i can say it will be effective for choosing lazarus/fpc over other languages and IDEs for programming newcomers.



i'm ready to start the job ,
if you know what the class or method is suitable for adding this feature
or you want help doing this  ;)
or even you have any comment about , i will be happy to know.


Best Regards.
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on October 22, 2015, 09:53:33 pm
Hi

I am the current maintainer of SynEdit in Lazarus.

   Welcome.

Nice to have someone interested in helping.


This should be done as a new subclass of TSynEditMarkup.

Colors can be added to TSynSelectedColor. look at frames and frame boundaries. Then the drawer will deal with them. (It needs testing for drawing in the middle of chars, which can happen if the line goes through a tab)

Look at SynEditMarkupWordGroup to see how to find Begin and end of blocks efficiently.


I will gladly add more help later.

-----------------
Things to start with

You need to understand:

http://wiki.lazarus.freepascal.org/SynEdit_Highlighter
How "ranges" are used in folding, and how they "level" can help finding blocks

Basics about what tokens are (this is how the HL splits the line). You probably will not need details.

http://wiki.lazarus.freepascal.org/SynEdit
logical and physical caret. this is essential.

Have a look at TSynSelectedColor and what it already provides

Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on October 22, 2015, 11:14:25 pm
this is also good http://wiki.lazarus.freepascal.org/Redesign_of_the_SynEdit_component

Not 100% up to date, and not 100% related, but an overview how things connect.
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on October 22, 2015, 11:30:34 pm
SynEditMarkupWordGroup  really is the start point. It is the code that highlights matching begin / end, if the cursor is on it

When you write your code, you will need to find all begin/end (try finally / repeat until / ...) pairs (iirc you can even find if/then this way).

Of course in
Code: Pascal  [Select][+][-]
  1. while a
  2. do
  3. begin
  4.   //...
  5. end;
  6.  
you then get a line for begin end, not starting at the while (same as code folding does).

So the first task is to find all those blocks in the current display. And it must be efficient, with folding there can be 1000 lines between top and bottom line.
For that you use the HL level info. (and later tricks like caching, but not to worry now)

When you have all the blocks surrounding the current line, then you know where to draw the lines. (Also when you have all blocks for line N, remember them, and if the next line is N+1 you can check which ones to re-use.)

There is a method PrepareMarkupForRow in which you should prepare this list.
--------------------

There will be calls to
    Procedure GetNextMarkupColAfterRowCol(const aRow: Integer;
                                          const aStartCol: TLazSynDisplayTokenBound;
                                          const AnRtlInfo: TLazSynDisplayRtlInfo;
                                          out   ANextPhys, ANextLog: Integer); virtual; abstract;


Find the first pos after aStartCol on which you need to draw a line, in your list of blocks  and return it.

and in GetMarkupAttributeAtRowCol
wait until that column is due. then return a markup with a frame on the left side only (and set the frameStartBound (search examples in other markup)


Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: AlexTP on October 23, 2015, 12:09:43 am
Martin Fr.

Look at your msg.

You write each sentence, almost each, on new line.

Even not new line.
Even on new paragraf.

This is irritating for usual reader. IMHO.

Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: kapibara on October 23, 2015, 01:22:55 am
On the contrary I think, it's very readable.
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on October 23, 2015, 01:48:20 am
reduced a few of them. But basically it is a bullet point list, without the markup
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: A-M on October 23, 2015, 09:08:31 am
Thank you guys for your attention
and specially thank martin for welcoming me
i'm reading the documentation and codes and will post a summary of my survey soon .
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: A-M on October 23, 2015, 10:00:14 am
Summary [updating]:

1-the whole job consist of two parts :
1-1-coloring the keywords with different colors on different depths that seems to be possible just by extending the Syntax Highlighter class
1-2-drawing the vertical lines behind the code blocks that seems to need editing the main TSynEdit class


2- TSynCustomFoldHighlighter is most complete highlighter class that consist Folding & Highlighting information together , TSynPasSyn that logically should be the Lazarus IDE highlighter extended from this class , this class has a property named "KeyAttri"(KeywordAttribute) of TSynHighlighterAttributes class that defined on SynEditHighlighter Unit;
and finally TSynHighlighterAttributes consist of some attribute including foreground color , background color and etc ...
As we need to coloring the Keywords on each depth with different colors we need "KeyAttri" to be an Array of  TSynHighlighterAttributes
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: molly on October 23, 2015, 12:42:25 pm
pardon the intrusion, you meant something similar like SynUniHighlighter (https://www.openhub.net/p/unihighlighter) or SynFacilSyn (https://github.com/t-edson/SynFacilSyn) ? (first seems hard to find, here (https://github.com/alrieckert/lazarus/tree/master/components/synunihighlighter) perhaps ?)
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on October 23, 2015, 12:44:20 pm
Unfortunately it is not the Highlighters job.

The highlighter is doing the basic parsing. Though the line is not 100% clear in this case. But the Markup is match better suited.

The Paint class paints the text token by token. It pulls them from the HL but they have a long way.

unit LazSytTextArea
Code: Pascal  [Select][+][-]
  1. function TLazSynPaintTokenBreaker.GetNextHighlighterTokenFromView(out
  2.   ATokenInfo: TLazSynDisplayTokenInfoEx; APhysEnd: Integer; ALogEnd: Integer): Boolean;
  3. ...
  4.   function MaybeFetchToken: Boolean; inline;
  5.   begin
  6. ....
  7.     while FCurViewToken.TokenLength = 0 do begin // Todo: is SyncroEd-test a zero size token is returned
  8.       Result := FDisplayView.GetNextHighlighterToken(FCurViewToken);
  9.  

FDisplayView is a TLazSynDisplayView

It comes from the Highlighter and passes through the TextBuffer and various Views (such as FoldedView). On each step it can be modified, or even tokens be added.

unit LazSytTextArea
Code: Pascal  [Select][+][-]
  1. function TLazSynPaintTokenBreaker.GetNextHighlighterTokenEx(out
  2.   ATokenInfo: TLazSynDisplayTokenInfoEx): Boolean;
  3. ...
  4.     FMarkupManager.GetNextMarkupColAfterRowCol(FCurTxtLineIdx+1,
  5.  

Calls the markup manager, to apply final markup.
This includes selection, visible whitespace-color, current word markup, ....

Now nested begin end can be seen as pascal grammar specific, but the kind of markup can work with any HL that has folding. It could work with LFM too.

So it really should be in a Markup.

Also you have markup on lines that between the keywords. The HL has no business to calculate them, and the painter should neither. This calculation goes into the Markup class.

--------------

The markup can and should find all blocks surrounding the line currently painted.
It can then:
- modify tokens for begin/end (as SynEditMarkupWordGroup  does)
- modify tokens on the line between, that need a vertical line.

--------------

When this works there may be a need to add things to the painter, because if a vertical line is dotted or dashed, it will not look good. A new fragment of it is painted for each text line, and that resets the dotted pattern.

This needs to be fixed in the painter, but this is for the very end.
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on October 23, 2015, 12:52:17 pm
For reference. This are the current markup classes:
  SynEditMarkup, SynEditMarkupHighAll, SynEditMarkupBracket, SynEditMarkupWordGroup,
  SynEditMarkupCtrlMouseLink, SynEditMarkupSpecialLine, SynEditMarkupSelection,
  SynEditMarkupSpecialChar,

And 99% of your code will be in a new sub class of TSynEditMarkup.

At this moment the only thing that goes outside is a fix in the painter for dotted vertical lines continued over several lines of text.
They still need to be drawn in individual bits per text line, in order to obey priority against other lines. The painter can use clipping to ensure they are joined correctly
I would not worry about that now and concentrate on the Markup only
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on October 23, 2015, 01:17:02 pm
There also is TLazSynEditNestedFoldsList which can help finding begin/end and other fold nodes in the HL.

It is used in ide/SourceSynEditor
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: A-M on October 23, 2015, 05:20:39 pm
Unfortunately it is not the Highlighters job.

The highlighter is doing the basic parsing. Though the line is not 100% clear in this case. But the Markup is match better suited.

The Paint class paints the text token by token. It pulls them from the HL but they have a long way.

i'm a bit confused from your post,may be because i have no background on TSynEdit
but i strongly believe at least half of work should be in HL
the below code as you know is GetTokenAttribute function in Pascal HL

Code: Pascal  [Select][+][-]
  1. function TSynPasSyn.GetTokenAttribute: TSynHighlighterAttributes;
  2. begin
  3.   case GetTokenID of
  4.     tkAsm: Result := fAsmAttri;
  5.     tkComment: Result := fCommentAttri;
  6.     tkIDEDirective: begin
  7.       FCurIDEDirectiveAttri.Assign(FCommentAttri);
  8.       FCurIDEDirectiveAttri.Merge(FIDEDirectiveAttri);
  9.       Result := FCurIDEDirectiveAttri;
  10.     end;
  11.     tkIdentifier: Result := fIdentifierAttri;
  12.     tkKey: Result := fKeyAttri;
  13.     tkNumber: Result := fNumberAttri;    
  14.     ...
  15.  

i think the work is easy as doing these steps on Pascal HL :
1-change fKeyAttri definition from TSynHighlighterAttributes to Array of TSynHighlighterAttributes .
2-Defining a variable for keeping current token depth that will increase on StartCodeFoldBlock and decrease on EndCodeFoldBlock.
3-finally doing the change on GetTokenAttribute like this :
tkKey: Result := fKeyAttri[CurrentTokenDepth];

why do you think this is not correct ?
i agree that there is a long way from HL to paint class but what is the problem ?

However i have no idea about drawing that vertical lines till now ...
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: A-M on October 23, 2015, 05:35:06 pm
pardon the intrusion, you meant something similar like SynUniHighlighter (https://www.openhub.net/p/unihighlighter) or SynFacilSyn (https://github.com/t-edson/SynFacilSyn) ? (first seems hard to find, here (https://github.com/alrieckert/lazarus/tree/master/components/synunihighlighter) perhaps ?)
you are welcomed to take a part in discussion
please look at the first post attachment to see what we want
i didn't found same thing on your links.
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Edson on October 23, 2015, 06:17:32 pm
1-the whole job consist of two parts :
1-1-coloring the keywords with different colors on different depths that seems to be possible just by extending the Syntax Highlighter class
1-2-drawing the vertical lines behind the code blocks that seems to need editing the main TSynEdit class

IMHO, the first part is entirely work for the highlighter.

SynFacilSyn have a mode of colour blocks using level of nesting. It does something like:

Code: Pascal  [Select][+][-]
  1. function TSynFacilSyn.GetTokenAttribute: TSynHighlighterAttributes;
  2. ...
  3.     case ColBlock of
  4.     cbLevel: begin  
  5.         Result.Background:=RGB(255- CodeFoldRange.CodeFoldStackSize*25,255- CodeFoldRange.CodeFoldStackSize*25,255);
  6.       end;
  7. ...
  8.  

I remember having some problems on getting nesting information from TSynCustomFoldHighlighter. IIRC it was not designed to manage the block information properly. It was designed just for folding.
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on October 23, 2015, 07:12:13 pm
Just because it is possible in the HL, doesnt mean it should be there.

Each feature should have its own class (or at least not be spread to more places than absolutely needed). That makes it more maintainable.
Not all of SynEdits code is like that yet, but I spent lots of times over the year to move into that direction. That is why even the highlight of the selected text is done in a markup class. It keeps the painting code more generic.

The vertical lines, have nothing to do with the HL at all. And the paint-class is generic to all languages, so the vertical lines must be done in a place in the middle.
So if one part of it can/should not be done in the HL, then why split it into different places, when it can be done all in one place?

If it should be included in the official release, then markup is the place to go.

Quote
Defining a variable for keeping current token depth that will increase on StartCodeFoldBlock and decrease on EndCodeFoldBlock.

This exists, and is accessible from the Markup. The foldlevel is explained on the wiki I linked.
I also gave you the name of a class that helps searching them.

Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on October 23, 2015, 07:22:07 pm
IMHO, the first part is entirely work for the highlighter.

I agree that it is easy to archive in the HL. Though it is also easy in Markup, as it would still be based on the same code.

As to "Should it be in the HL", that depend on the bigger design of SynEdit, and the exact definition of what it should archive.

If the goal was to write code that
- does only the keywords, bet NOT the vertical lines
- that can only work on pascal, and due to its properties can never possible work on other languages
then the HL is the place.

But with the exception of incorrect implementation, there is nothing to stop this from working on lfm, html, .... Even if not implemented today (and that is fine), it can be easily extended later (without making copies of the same code in every HL)

Also since it needs the vertical lines, it can not go into the HL. SynEdit (the lazarus variant) had the markup classes specially designed for this.
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: A-M on October 23, 2015, 11:13:21 pm
Ok martin due to your experience in this field your opinion should be more accurate , so i'm going on TSynEditMarkup way ...
unfortunately i didn't found any sample or good explained document about markup classes till now

PS: i don't know why ! but i think it will takes just a few hours for you to add this excellent feature to SynEdit  ;)
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on October 23, 2015, 11:37:41 pm
Well it is on my list, and has been there for some long time.  Right now I have other priorities.

One of the issues is, that I would only do this if I have time to optimize it, so the amount of searching can be minimized.
On a desktop it does not matter, on a laptop (used by many) it drains the battery.

But I accept a patch without this optimization.

Code: Pascal  [Select][+][-]
  1.     Procedure PrepareMarkupForRow(aRow : Integer); virtual;
  2. // called before each row that needs painting.  May only be part of the screen, and skips folded lines
  3. // normally top down, but if you relay on it, verify it in code (compare to last line no, if you stored that)
  4.  
  5.  
  6.     Procedure FinishMarkupForRow(aRow : Integer); virtual;
  7. // when the line is painted
  8.  
  9.     Procedure EndMarkup; virtual;
  10. // when all lines are painted / reset any cache, that is not otherwise reseted
  11.  
  12.     Procedure GetNextMarkupColAfterRowCol(const aRow: Integer;
  13.                                           const aStartCol: TLazSynDisplayTokenBound;
  14.                                           const AnRtlInfo: TLazSynDisplayRtlInfo;
  15.                                           out   ANextPhys, ANextLog: Integer); virtual; abstract;
  16. // set ANextPhys, ANextLog to the next pos at which you need to change color
  17. // start or end of "begin"
  18. // pos of vertical line
  19.  
  20.     Function  GetMarkupAttributeAtRowCol(const aRow: Integer;
  21.                                          const aStartCol: TLazSynDisplayTokenBound;
  22.                                          const AnRtlInfo: TLazSynDisplayRtlInfo) : TSynSelectedColor; virtual; abstract;
  23. // return markupinfo with desired color from aStartCol to GetNextMarkupColAfterRowCol
  24. // the colors will be merged according to their priorities
  25.  
  26. // I need to remind myself, or you search existing code, when setting frame start/end (vertical line) you need to call MarkupInfo.SetFrameBoundsLog
  27.  
  28.  

priorities: check out the ide color config

In order to have many colors add more markupinfos, then return the correct one in GetMarkupAttributeAtRowCol(

try and understand SynEditMarkupWordGroup. this is essential.

Then look at TLazSynEditNestedFoldsList  it can help you find all begin end on current line and all the ones you are nested in.


Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Edson on October 24, 2015, 06:04:42 pm
Personally, I'm no sure if many colors on the editor, can help on visualizing better the code. I prefer to have background colors.

Vertical lines, can be useful. I see it like an extension of the folding marks.

I like the way Notepad++ highlight the color of the folding lines, according to the cursor position.
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: A-M on October 24, 2015, 07:36:17 pm
Personally, I'm no sure if many colors on the editor, can help on visualizing better the code. I prefer to have background colors.

Vertical lines, can be useful. I see it like an extension of the folding marks.

I like the way Notepad++ highlight the color of the folding lines, according to the cursor position.

there is really not many colors ... a color for a certain block depth & most of the time there is maximum 4 colors on the screen for keywords
and the colors can be very similar if you want

did you worked with delphi when colorized by CNPack ? its really amazing and your mind will not be weary even after several hours of coding
i really don't know any serious and not retired  delphi programmer around me who not using CNPack

vertical lines are what the modern & professional IDE like IntelliJ equipped with and i believe the small area of brackets in languages other than pascal is not suitable for coloring the code blocks ...
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: A-M on October 24, 2015, 08:15:24 pm
I would confess that the job is harder than i thought ....
there is a lot of records , classes and functions that needs a long time to understand
but i really want this feature on Lazarus so there is no surrender

I have to ask you martin for explanation on each one but please don't look at this as a waste of time because i will integrate all as a Wiki Page , if God will.
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on October 24, 2015, 10:01:46 pm
The 3 main steps (in that order) / use latest svn


Well play around with the markup to see how it works, subclass it.
- Add it in the synedit constructor (find the others) line 2085
     fMarkupManager.AddMarkUp(....);
- add a color (background/frame) to its YourMarkup.markupinfo
- return it conditonal (eg odd lines) as a result of GetMarkupAttributeAtRowCol
- start setting AnextLog/Phys column in GetNextMarkupColAfterRowCol (dependend on AstartCol), and return markupinfo in GetMarkupAttributeAtRowCol dependend on color.

That should give you the basic feeling of how it works.
btw Markupmanager should probably be public in SynEdit.

You need more than one color? Add more MarkupInfo objects.

Then you can check how to only return a vertical line (frame on left only) SetFrameBound()

-----------
Once you know the markup

to find token on the current line
Code: Pascal  [Select][+][-]
  1. FFoldNodeInfoList := AHighlighter.FoldNodeInfo[ALine];
attention the result must be freed using "FFoldNodeInfoList.ReleaseReference()"

This is used in SynEditMarkupWordGroup;

Look at it, and explore the classes/records.

Node that there are 2 foldlevel
1) level for items that are configured to be foldable by the user (not of interest)
2) level for all items. (that you need)

This allows you to add colors to the begin/end etc.

-----------
look at ide/sourcesyneditor
TIDESynEditor.SrcSynCaretChanged
Code: Pascal  [Select][+][-]
  1.       List := TextView.FoldProvider.NestedFoldsList;
  2.  
this returns all the begin from previous lines, that the current line is nested in, so all the x pos for vertical lines

Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on October 24, 2015, 10:10:44 pm
I added a public property in svn/trunk
SynEdit.MarkupManager

So you can drop a SynEdit on a form, subclass your own Markupclass, and play with it.
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: A-M on October 24, 2015, 11:02:44 pm
thank you for your explanations , i will ask my questions ...

I added a public property in svn/trunk
SynEdit.MarkupManager
i got it
why is this private in normal releases ?
Quote
    property MarkupManager: TSynEditMarkupManager read fMarkupManager;
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: A-M on October 24, 2015, 11:09:51 pm
Q1: what the caret really are in SynEdit ? text blinking cursor as written on wiki ?
or can we say simply this is a method for specifying a point on TSynEdith ?
if its just for cursor position so can we say this is not related to our MarkUp class ?

Q2: in below procedure its just three keywords (tokens) to find
Code: Pascal  [Select][+][-]
  1. procedure TSynEditMarkupWordGroup.FindMatchingWords(LogCaret: TPoint;
  2.   out Word1, Word2, Word3: TWordPoint);

it seems enough for drawing vertical lines but what about coloring the keywords on each depth ?
it will be unlimited keywords (tokens) that can be related to each other (on the same depth)...
similar to below code


Code: Pascal  [Select][+][-]
  1.      
  2. while True do
  3.      if x then
  4.      begin
  5.      //...
  6.      end
  7.      else if y then
  8.      begin
  9.      //...
  10.      end
  11.      else ...
  12.  

what is the solution ? using a dynamic array for keeping related keywords (tokens) ? or what ?

Q3: how we want to draw that vertical lines ? using GetMarkupAttributeAtRowCol ? if son i think this will not be a continuous line ...
or directly using SynEdit's Canvas for drawing the line ? or what ?
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: A-M on October 24, 2015, 11:24:56 pm

to find token on the current line
Code: Pascal  [Select][+][-]
  1. FFoldNodeInfoList := AHighlighter.FoldNodeInfo[ALine];
attention the result must be freed using "FFoldNodeInfoList.ReleaseReference()"

This is used in SynEditMarkupWordGroup;

Look at it, and explore the classes/records.

as i seen there is no such a thing in SynEditMarkupWordGroup even in svn ...
Are you sure ?
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: A-M on October 24, 2015, 11:33:16 pm
look at ide/sourcesyneditor
TIDESynEditor.SrcSynCaretChanged
Code: Pascal  [Select][+][-]
  1.       List := TextView.FoldProvider.NestedFoldsList;
  2.  
this returns all the begin from previous lines, that the current line is nested in, so all the x pos for vertical lines

if we draw the vertical line from (x,y1) to (x,y2) ? what about y ?
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on October 25, 2015, 02:54:32 am
Quote
why is this private in normal releases ?
It will be public in 1.6 release. I didnt make it public when I first implemented it, in case it would still change. Then I forgot.

Quote
Q1: what the caret really are in SynEdit ? text blinking cursor as written on wiki ?
or can we say simply this is a method for specifying a point on TSynEdith ?
if its just for cursor position so can we say this is not related to our MarkUp class ?

Normally it refers to the blinking line. Put the term is often used for variables that hold a XY coordinate.
Actually those variable are misnamed. "LogCaret" in this case should be "LogXY", since it is not a caret. (You got me there)

Quote
it seems enough for drawing vertical lines but what about coloring the keywords on each depth ?
it will be unlimited keywords (tokens) that can be related to each other (on the same depth)...
similar to below code
I suggest to get back to this when you done:
- getting to know TSynEditMarkup
- highlight keywords on each line

But as I wrote in my 3rd point TextView.FoldProvider.NestedFoldsList; / TLazSynEditNestedFoldsList  will do help you.

This list, if created for a line will find ALL blocks that are open at the begin of that line.

Actually with
Code: Pascal  [Select][+][-]
  1. List.IncludeOpeningOnLine := False;
it might help for the current line too.

This should include those, that open on the current line.
You still need those that close at the current line.

Code: Pascal  [Select][+][-]
  1. procedure TIDESynEditor.SrcSynCaretChanged(Sender: TObject);
  2.   function RealTopLine: Integer;
  3.   begin
  4.     Result := TopLine - TSourceLazSynSurfaceManager(FPaintArea).TopLineCount;
  5.   end;
  6. var
  7.   NodeFoldType: TPascalCodeFoldBlockType;
  8.   List: TLazSynEditNestedFoldsList;
  9. begin
  10.   if (not FShowTopInfo) or (not HandleAllocated) or (TextView.HighLighter = nil) then exit;
  11.   if FSrcSynCaretChangedLock or not(TextView.HighLighter is TSynPasSyn) then exit;
  12.  
  13.   if TextView.HighLighter.NeedScan then begin
  14.     FSrcSynCaretChangedNeeded := True;
  15.     // USING TLazSynEditNestedFoldsList   for this line
  16.  

will give you:
* from line 1 procedure
* from line 9 begin
* from line 13 begin

I am not sure it includes the "then". But that is covered by the begin in that line.

that are the ones for which you need a vertical line on this text line.

Doing so for every line over and over again. (And that is where caching may be needed. / I do not remember, maybe TLazSynEditNestedFoldsList actually does that already)

Quote
Q3: how we want to draw that vertical lines ? using GetMarkupAttributeAtRowCol ? if son i think this will not be a continuous line ...

Set the left frame side. If testing shows that is not enough then we need a new attribute on MarkupInfo : TSynSelectedColor.
Drawing directly on the canvas would not support the priority and alpha info on TSynSelectedColor.

Having said that, the drawer will need a fix for continuous dotted lines, but I suggest to discuss that at the end.

Quote
as i seen there is no such a thing in SynEditMarkupWordGroup even in svn ...
Are you sure ?

As I wrote:
look at ide/sourcesyneditor
TIDESynEditor.SrcSynCaretChanged

Quote
if we draw the vertical line from (x,y1) to (x,y2) ? what about y ?

sorry not sure what you mean.
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on October 25, 2015, 02:03:12 am
from sourcesynEditor
Code: Pascal  [Select][+][-]
  1.       List := TextView.FoldProvider.NestedFoldsList;
  2.       List.ResetFilter;
  3.       List.Clear;
  4.       List.Line := CaretY-1;
  5.       List.FoldGroup := FOLDGROUP_PASCAL;
  6.       List.FoldFlags := [sfbIncludeDisabled];
  7.       List.IncludeOpeningOnLine := False;
  8.  
  9.       InfCnt := List.Count;
  10.       for i := InfCnt-1 downto 0 do begin
  11.         NodeFoldType := TPascalCodeFoldBlockType({%H-}PtrUInt(List.NodeFoldType[i]));
  12.         if not(NodeFoldType in
  13.            [cfbtClass, cfbtClassSection, cfbtProcedure])
  14.         then
  15.  

this goes through the list and look for procedures and class and public/private
You can do the same, but check for begin, while, repeat, ....

Then check the info you can get from the object. Code navigation should show it.

Note the list returns pointers because it could be used for none pascal HL too, therefore the typcasts.

--------
List.Line
Quote
property Line: TLineIdx read FLine write SetLine;
"Idx" that means almost always 0 based.

While a name ending in ...Pos should be 1 based.

there are helpers in SynEditMiscProcs
function ToIdx(APos: Integer): Integer; inline;
function ToPos(AIdx: Integer): Integer; inline;

if you need to go from 0 to 1 based or vice versa, then use them. That is much clearer than  Y+1 or Y-1
They are new, so lots of code does not yet use them
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: x2nie on October 25, 2015, 06:34:12 am
Personally, I'm no sure if many colors on the editor, can help on visualizing better the code. I prefer to have background colors.

Vertical lines, can be useful. I see it like an extension of the folding marks. ...

there is really not many colors ... a color for a certain block depth & most of the time there is maximum 4 colors  ...


Personally, I'am sure many color helped me on understanding pascal codes better.
Specially when I work in too many deep of nested begin..end (see the attachment for sample)


Hey, I am counting and there are 6 colors used. so, while there 6 colors (last color) exceeded , that 6 colors used again from beginning.
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: x2nie on October 25, 2015, 07:19:31 am
Anyway, still about this colouring topic, current Lazarus is better (smart) than newest version of CnPack
when deal with {$IFDEF}.
It's important aspect, because ignoring this part, the whole work of nested-block-coloring will just guides programmer into wrong logic/algorithm. (see the screenshot for detail)


I also attached 2 file that can be used for later test.(not allowed by this forum's robot)
The gr32.pas and gr32_resamplers.pas is part of graphics32 library. it can be downloaded from such from https://searchcode.com/codesearch/view/62389395/ (https://searchcode.com/codesearch/view/62389395/)






Anyway, if somebody could get any progress, can I collaborate? perhaps you must create another repository in sourceforge/svn with simple application +1SynEdit ?  8-)
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: x2nie on December 01, 2015, 11:21:09 am
..
There is a method PrepareMarkupForRow in which you should prepare this list.
--------------------

There will be calls to
    Procedure GetNextMarkupColAfterRowCol(const aRow: Integer;
                                          const aStartCol: TLazSynDisplayTokenBound;
                                          const AnRtlInfo: TLazSynDisplayRtlInfo;
                                          out   ANextPhys, ANextLog: Integer); virtual; abstract;


Find the first pos after aStartCol on which you need to draw a line, in your list of blocks  and return it.

and in GetMarkupAttributeAtRowCol
wait until that column is due. then return a markup with a frame on the left side only (and set the frameStartBound (search examples in other markup)


Hi Mr. Martin, I did exactly what you said. It done nicely. (see attachment)
And yes, I agree that it should be in TSynEditMarkup, so it can also be used later in any TSynHighliter; rather than dedicated pascal highlighter only.


Now, the problem is it only care about the folding signal.
So, how we can continue this progress to care of pascal keywords (while, do, if, then .. ) ?


------
Anyway, I have different approach:
I currently don't care about begin-end pairs, because I don't want to draw the vertical lines yet. (Sure I will do that later.)
Instead, I only care about the level of any "folding" found in the line. <--- this level info is already calculated, so I don't do any calculation.
and I simply give a color related to the level:


Code: Pascal  [Select][+][-]
  1.     if sfaOpen in TmpNode.FoldAction then
  2.       lvl := TmpNode.FoldLvlStart
  3.     else
  4.       lvl := TmpNode.FoldLvlEnd;
  5.     ColorIndex := lvl mod (length(Colors));    
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on December 01, 2015, 11:44:22 pm
Cool, I will try to find some time to review it in more detail.

One thing I noted: You only look for the first TmpNode in "TmpNode := NodeList[ i ];". But there can be more than one "begin" in a line. Is that intended?
You could search for the next one in GetNextMarkupColAfterRowCol, once you are past the first. You would need to keep the list, release it in EndMarkup (and in PrepareRow, if none nil, or if in PrepareRow you got it already, just change the line.).
This would also fix the not marked "end" in your screengrab.

Quote
Now, the problem is it only care about the folding signal.
So, how we can continue this progress to care of pascal keywords (while, do, if, then .. ) ?

if/then/else/; is actually available (depends on filter settings), It is not so much for folding, but the HL must keep track of nested if, in order to differentiate between a case-else and if-else.

It is disabled. In the IDE it can be turned on via the ide options, but in SynEdit you need to add some code, I need to search myself. try "NodeList.FoldFlags:= [sfbIncludeDisabled]" instead

-----------
"while" should ideally not be added as fold-level.  Adding more items to the fold-level increases the resources the HL will need.

Though if it is the ONLY one missing it may have do be discussed.

How would/should a "while" look? without begin/end it has no end-token (well the ";" terminates it)?
Of course once you have vertical lines, there is a use.

Is "while" the only one missing?

How much nesting support does it need? All while (if without begin/end) end in the same place, at the ";"
Code: Pascal  [Select][+][-]
  1.   while a do
  2.     while b do
  3.       foo();
-----------

if an "if" or "while" has a begin/end: will they both highlight?

If you have a begin block, you can check the outer block.

But maybe better, the HL has sfaMarkup to indicate word pair highlight. It may introduce sfaMarkupOutline. Then the HL can determine which nodes to outline and which not. (so the markup will stay independent)

For info, each nodes has flags
Code: Pascal  [Select][+][-]
  1.   TSynFoldAction = (
  2.                      sfaOpen,         // Any Opening node
  3.                      sfaClose,        // Any Closing node
  4.  
  5.                      sfaFold,         // Part of a fold- or hide-able block (FoldConf.Enabled = True)           - excludes one=liners for FoldFold, as they can not fold
  6.                      sfaFoldFold,     // Part of a fold-able block (FoldConf.Enabled = True / smFold in Modes)  - excludes one=liners / only opening node, except ifdef/region (todo: maybe both?)
  7.                      sfaFoldHide,     // Part of a hide-able block (FoldConf.Enabled = True / smHide in Modes)  - includes one=liners / only opening node, except ifdef/region (todo: maybe both?)
  8.  
  9.                      sfaMultiLine,    // The closing node is on an other line
  10.                      sfaSingleLine,   // The closing node is on the same line (though the keyword may be on the next)
  11.                      // //sfaSingleLineClosedByNext
  12.                      sfaCloseForNextLine,  // Fold closes this line, but keyword is on the next (e.g. "var" block)
  13.                      sfaLastLineClose,     // Fold is incomplete, and closed at last line of file
  14.  
  15.                      sfaDefaultCollapsed,
  16.                      sfaMarkup,   // This node can be highlighted, by the matching Word-Pair Markup
  17.                      sfaInvalid,  // Wrong Index
  18.  
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: x2nie on December 02, 2015, 01:29:59 pm
Hi Mr. Martin


* the single line 'begin~end' has been resolved.
* The [sfbIncludeDisabled] doesn't work, because No method ables to make it applicable to NodeList.
* Yes, nested "while-do--while-do;" can be painted by one color. Therefore "if-then-else--if-then-else" too;
The reason is because both is just one pascal statement ( = closed with with one " ; ")
But, nested routine isn't. Meaning...
Code: Pascal  [Select][+][-]
  1. procedure funcParent();
  2.    procedure funcChild();
  3.    begin
  4.       ...
  5.    end;
  6. begin
  7. ...
  8. end;
The begin+end part of a routine should be painted in differently color compared to begin+end part of it's children.
It's needed for notice us that the begin+end block of child is not yet the main block of the parent function.


* I got a few more progress, I can now coloring the several pascal keyword without any call to specific (pascal) highlighter.


----
I also found another problem: the higlighter reports inconsistency in if~then block. see the attachment.
I don't know why the 'if' is painted in similar level with 'begin', and in another line it is painted differently.


----
I am now working in drawing vertical lines.
I found that most efficient way to draw these lines is by: just reading the cache.
so I will move the PrepareMarkupForRow's content into DoTextChanged.
Further I plan the cache should be progresif: continue find the 'end' part of folding sytem only if needed.
such when user scroll down the page. in other word: we don't need to build the cache of the whole lines, only the was visible area will be calculated.


----
I also play with the TSynFoldAction. The cons is it work only with 'and' mode, while I need the 'or' mode.
So, NodeList.ActionFilter := [sfaSingleLine,sfaMultiLine];  (or similar combination)  doesn't work.
THe only way I can reach is by call filtering twice. It's not a problem for now, I just dream that there are a simpler way. 8-)


The funny thing: It's running well: NodeList.ActionFilter := [];  8-)
So, I keep this way.


-----
The cons of current TSynPascalSyn is : It doesn't report the "with~do" as fold signal, while the "repeat~until" works.

Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: x2nie on December 02, 2015, 01:49:29 pm
if an "if" or "while" has a begin/end: will they both highlight?
Yes it is. Maybe they (if-then-begin-end-else-begin-end) should be painted in one color per level.
But, configurable (highlighted or not) may be better for other user.

If you have a begin block, you can check the outer block.

But maybe better, the HL has sfaMarkup to indicate word pair highlight. It may introduce sfaMarkupOutline. Then the HL can determine which nodes to outline and which not. (so the markup will stay independent)
sfaMarkupOutline is not found in current lazarus trunk, okay?
I think implement it will solve the problem (my previous post).
It will also make the synedit folding system more flexible and clean.

---
Sorry for was not understand some parts of your post. Therefore I report problem that already has given clues by you in several post.
I only understood them after trying to code my self. >:D
And while reading again your post, they (what you said) seem clearer for me.  ;) :D
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: x2nie on December 02, 2015, 02:19:34 pm

--------------

When this works there may be a need to add things to the painter, because if a vertical line is dotted or dashed, it will not look good. A new fragment of it is painted for each text line, and that resets the dotted pattern.

This needs to be fixed in the painter, but this is for the very end.

At this moment the only thing that goes outside is a fix in the painter for dotted vertical lines continued over several lines of text.
They still need to be drawn in individual bits per text line, in order to obey priority against other lines. The painter can use clipping to ensure they are joined correctly
I disagree.
I have different approach that doesn't require the change of painter:
* We can use "GetMarkupAttributeAtRowCol()" as a command instruction to painter, and let the only painter do painting work.
* what we send to painter is just as usually (X,X2, colorForeground, backgroundColor, framecolor).
The trick is we send instruction to the painter to paint only the left border on given highlighted area.


* but the only problem is: how to send the "bsLeft" part of  TLazSynBorderSide ? Is the TSynEditMarkup.MergeMarkupAttributeAtRowCol the right place to do that?  :o


Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on December 02, 2015, 03:29:55 pm
Quote
* The [sfbIncludeDisabled] doesn't work, because No method ables to make it applicable to NodeList.
property FoldFlags: TSynFoldBlockFilterFlags

Quote
* Yes, nested "while-do--while-do;" can be painted by one color. Therefore "if-then-else--if-then-else" too;
Quote
the higlighter reports inconsistency in if~then block. see the attachment.
if-then-else are 2 or even 3 blocks. The "else" is in both.
- if: open
- then: close / open another
- else: close / open another
- ; end or other closing reason: close

So each of them has 2 entries in  the node-list.

Quote
I am now working in drawing vertical lines.
I found that most efficient way to draw these lines is by: just reading the cache.
What cache?
Quote
so I will move the PrepareMarkupForRow's content into DoTextChanged.
DoTextChanged can be called a lot more often. THat is not a good idea. It can also be called for of screen changes.
It is meant to be used to invalidate cached data, usually be setting a flag (which does not cost time) "InvalidateNeeded := true;"

Quote
I also play with the TSynFoldAction. The cons is it work only with 'and' mode, while I need the 'or' mode.
So, NodeList.ActionFilter := [sfaSingleLine,sfaMultiLine];  (or similar combination)  doesn't work.
Then do not filter at all, and filter the result you read. Or suggest a way to add it to the list, and a patch can be discussed.
Quote
The funny thing: It's running well: NodeList.ActionFilter := [];  8-)
So, I keep this way.
Yes, IIRC filter only filters, if at least one filter is set.

Quote
The cons of current TSynPascalSyn is : It doesn't report the "with~do" as fold signal, while the "repeat~until" works.
ok, so "with" and "while" are both missing.

I currently am a bit time pressed, so I have to answer that later.


Quote
Quote from: Martin_fr on December 01, 2015, 11:44:22 pm

    if an "if" or "while" has a begin/end: will they both highlight?

Yes it is. Maybe they (if-then-begin-end-else-begin-end) should be painted in one color per level.
But, configurable (highlighted or not) may be better for other user.
the HL can be modified to deliver correct info for that, later more.

that is what sfaMarkupOutline is about. It needs to be ADDED. then the HL can set it as needed. (inclusive user conf)

more later

Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on December 02, 2015, 03:37:28 pm
When this works there may be a need to add things to the painter, because if a vertical line is dotted or dashed, it will not look good. A new fragment of it is painted for each text line, and that resets the dotted pattern.

This needs to be fixed in the painter, but this is for the very end.

At this moment the only thing that goes outside is a fix in the painter for dotted vertical lines continued over several lines of text.
They still need to be drawn in individual bits per text line, in order to obey priority against other lines. The painter can use clipping to ensure they are joined correctly
I disagree.
I have different approach that doesn't require the change of painter:
* We can use "GetMarkupAttributeAtRowCol()" as a command instruction to painter, and let the only painter do painting work.
* what we send to painter is just as usually (X,X2, colorForeground, backgroundColor, framecolor).
The trick is we send instruction to the painter to paint only the left border on given highlighted area.
Yes you use the frame left site.

What I meant is, if you use a dotted line, the painter will sometimes join it incorrectly causing tiny interruptions. But that can be fixed later.


Quote
* but the only problem is: how to send the "bsLeft" part of  TLazSynBorderSide ? Is the TSynEditMarkup.MergeMarkupAttributeAtRowCol the right place to do that?  :o

You sent it from GetMarkupColAtRowCol, like the color. Set the frame color and frame side, and check code in the other markup. you need to set
MarkupInfo.SetFrameBoundsLog(x1, x1)

Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: x2nie on December 03, 2015, 12:56:38 am
Quote
I am now working in drawing vertical lines.
I found that most efficient way to draw these lines is by: just reading the cache.
What cache?
Quote
so I will move the PrepareMarkupForRow's content into DoTextChanged.
DoTextChanged can be called a lot more often. THat is not a good idea. It can also be called for of screen changes.
It is meant to be used to invalidate cached data, usually be setting a flag (which does not cost time) "InvalidateNeeded := true;"


Well, that means we still use PrepareMarkupForRow?, and DoTextChanged  was not the right place to save the open~close--open~close folding list through several lines?
IMHO, Then the problem (with current version of SynEdit) is: identifiying of verticals line couldn't be efficient.

You know, in TSynEditMarkupWordGroup we search at least one pair of begin~end.
But in my TSynEditMarkupFoldColors, below sample require at least 5 pair begin~end for getting the correct position of it's 5 vertical lines:

| |    | | | HorzEntry := EMPTY_ENTRY;
(see attachment for complete lines)

Sure, we can do searching these 5 pairs on PrepareMarkupForRow, but AFAIK its not efficient in a long procedure/code.
In other words, why we should repeat finding all pairs per  line, while we can do that once per screen changes (DoTextChanged) ?
I choose DoTextchanged because it is the only chance before iteration of each line.
DoTextChanged is also paired with MarkupFinish(); Maybe I need something new like MarkupStart() that works outside the iteration of painting each lines.
Simply: When DoTextChangedMarkupStart is not the right place, IMHO PrepareMarkupForRow is even worst.


I think any other TSynMarkup is useless for comparing with TSynEditMarkupFoldColors's job
Thats why I planned that special case.


-----
But, wait...
There is another possible choice, which may best to do:
If PrepareMarkupForRow is really the only chance to identify the position of 5 vertical lines,
so it can be very efficient if there was no calculation/searching the pairs at all
in PrepareMarkupForRow.



How it will be done?
The highlighter (TSynCustomFoldHighlighter) should able to provide those 5 vertical lines's position !  8-) >:D
This way, the job done in PrepareMarkupForRow by simple TempNode := NodeList[ i ];
The real calculation is in TSynCustomFoldHighlighter (or descendant) which is the correct class who know how to.


Further, for simplify the TSynCustomFoldHighlighter's filter when deal without vertical-lines's requirement,
 I guess it should be something config, like : FoldFlag := [sbfIncludeDisable, sbfIncludeParents] ?


Quote
What cache?
The NodeList thats done calculated outside the PrepareMarkupForRow, that is cache in my mind.
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on December 03, 2015, 02:26:15 am
Quote
Well, that means we still use PrepareMarkupForRow?, and DoTextChanged  was not the right place to save the open~close--open~close folding list through several lines?
Yes, even so that I have not yet seen what is in DoTextChanged.


Quote
IMHO, Then the problem (with current version of SynEdit) is: identifiying of verticals line couldn't be efficient.

It should. Look at the word-pair (tripplet) markup.

Search of a matching fold node is very efficient. You can find the correct line without looking into the line, simply by looking at the level.
If you search ends for several begin, you know that the next end is further down, so you can continue in the found line.

You could cache this info (and invalidate in DoTextChanged). But you may need a real big cache. there could be several 100 visible lines. (small font, big screen).
Or there could be a lot of folded lines, so the last line visible in the window is 5000 lines below the first (4950 lines folded).

You definitely should cache between lines, during one screen refresh/paint. That is if you calculated for line 390, then you cache because likely 391 is next (unless folded). Cache is then cleared in EndMarkup.

Look at TLazSynEditNestedFoldsList. This list provide ALL open fold nodes at the start of a line
Code: Pascal  [Select][+][-]
  1. procedore foo,
  2. var
  3.   i: integer;
  4. begin
  5.   for a := 1 to 2 do begin
  6. // TLazSynEditNestedFoldsList will provide both begin, and the procedure, but not the var
  7.  
  8.  

The searching of matching end nodes could be added (not yet there) to this list. It is useful in many places.

When the first PrepareRowFor... is called, you get all open blocks (open from previous lines) from the list.
During the row you modify as required.

If rows are skipped (folded), then you ask TLazSynEditNestedFoldsList again. (you can merge with what you know from the above rows)

from markup
Code: Pascal  [Select][+][-]
  1. function TSynMarkupHighIfDefLinesTree.GetHighLighterWithLines: TSynCustomFoldHighlighter;
  2. begin
  3.   Result := FHighlighter;
  4.   if (Result = nil) then
  5.     exit;
  6.   Result.CurrentLines := FLines;
  7. end;
  8.  
  9. function TSynMarkupHighIfDefLinesTree.CreateOpeningList: TLazSynEditNestedFoldsList;
  10. begin
  11.   Result := TLazSynEditNestedFoldsList.Create(@GetHighLighterWithLines);
  12.   Result.ResetFilter;
  13.   Result.Clear;
  14.   //Result.Line :=
  15.   Result.FoldGroup := FOLDGROUP_PASCAL = ;
  16.   Result.FoldFlags := [sfbIncludeDisabled];
  17.   Result.IncludeOpeningOnLine := False;
  18. end;
  19.  

also used in ide/sourcesynEditor.pas
TIDESynEditor.SrcSynCaretChanged

Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on December 03, 2015, 02:29:33 am
Sorry, before my last post I mixed it up, and thought the list you used and TLazSynEditNestedFoldsList  where the same.
hence my post about sfbIncludeDisabled
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: x2nie on December 03, 2015, 06:20:11 am
Yes! you right.
everything can be done in PrepareMarkupForRow();


Now, the problem goes to TSynPascalSyn:
* The begin~end level of "if" is seem wrong.
* But, the begin~end level of "else" is correct.


Any idea ?


Another problem:
* I can not set left-only-border. The 4 side border always drawn.
   its because GetMarkupAttributeAtRowCol ( you told as GetMarkupColAtRowCol) is only work with TSynSelectedColor, while we need TSynSelectedColorMergeResult.
I have no clue how to instruct painter to draw only the left side.


Screenshots are only deal TLazSynEditNestedFoldsList.
I temporary disable the TLazSynFoldNodeInfoList (NodeList). But they are yet beautiful enough for now.

------
The code can be downloaded:
web: https://github.com/x2nie/syneditmarkupnestedcolors
svn: https://github.com/x2nie/syneditmarkupnestedcolors
git: https://github.com/x2nie/syneditmarkupnestedcolors
zip: https://github.com/x2nie/syneditmarkupnestedcolors/archive/master.zip

Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: x2nie on December 03, 2015, 06:39:02 am
Further more progress / bugfix:
if-begin-end vs else-begin-end coloring bug was my fault !! :-[  I've resolved that.


Now, the problem of nested-color is the ChildProcedure-begin-end. It should has different level than the MainProcedure-begin-end.
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on December 03, 2015, 06:44:17 am
Code: Pascal  [Select][+][-]
  1. TSynSelectedColor.FrameEdges := sfeLeft[
  2. /code]
  3. Should have been an enum, but isnt....
  4.  
  5. [quote]Now, the problem goes to TSynPascalSyn:
  6. * The begin~end level of "if" is seem wrong.
  7. * But, the begin~end level of "else" is correct.[/quote]
  8.  
  9. I had  a quick look. the issue is that there is one block from
  10. "if" to "then"
  11. and a 2nd block from
  12. "then" to "else"
  13.  
  14. or if there is no "else" it goes from "then" to ";"
  15.  
  16. But there is no block from "else" to ";". (that wasnt needed to get the "case a of 1: write ELSE read end;"
  17.  
  18. Not yet sure how to fix it best.
  19.  
  20. Also it currently the case there is actually no gurantee of monotonic increases.
  21. If the HL would need to keep track of something else, it might count up.
  22.  
  23. It can be added as extra counter, but that is probably more costly than counting it in the markup.
  24.  
  25. On the other hand we will need the else (like the "while") so maybe....
  26.  
  27. I have to come back on this. Might be a bit...
  28.  
  29. --------------
  30. But I do see you a making create progress :)
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on December 03, 2015, 06:46:51 am
Further more progress / bugfix:
if-begin-end vs else-begin-end coloring bug was my fault !! :-[  I've resolved that.


Now, the problem of nested-color is the ChildProcedure-begin-end. It should has different level than the MainProcedure-begin-end.

It will be if you enable folding for "procedure"

Code: Pascal  [Select][+][-]
  1. SynEdit.FoldConfig[ord(cfbtProcedure)].Enabled := true

Or use NestLvlStart instead of FoldLvlStart, but that may increase too fast.

So you may need to keep your own count, when you go through the nodes, and decide which ones to use
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: x2nie on December 03, 2015, 10:01:46 am
Hi Mr. Martin,


There is no different >> self.SynFreePascalSyn1.FoldConfig[ord(cfbtProcedure)].Enabled := True/False;


Anyway, I have good news:
the vertical lines finally drawn properly !  8-)
It is now dejavu. Very similar to CNPack.


But I found some bug made by me. Such as when a "begin" moved to the left (by deleting trailing space).
I will fix them later.


----
My priority (and curiosity) is now going to the highlighter it self.
I wonder, why SynHighlighterLFM doesnt reports any nested folding, sure the nested folding is enabled (I can click in the gutter) ?


I also  include the FoldHL.pas, it is the simplest highlighter I can found from the SynEditTutorial.
But it also doesn't reports any nested folding which can be used by SynEditMarkup.
What's I should add/modif to reach what SynHighligterPascal had has?  :o


----

The code can be downloaded:
svn: https://github.com/x2nie/syneditmarkupnestedcolors
git: https://github.com/x2nie/syneditmarkupnestedcolors
zip: https://github.com/x2nie/syneditmarkupnestedcolors/archive/master.zip

Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on December 04, 2015, 03:23:16 am
I have to get back on those questions.

Nice work, just downloaded.
-----------------------------
You need to add invalidation calls.

Add or remove space in front of a "begin" (then resize the editor to invalidate)


If markup changes for any line, then self.InvalidateSynLines(firstline, last);
must be called.

And that should happen before painting, which also means before GetNextMarkup......

So I was wrong on some of the previous info.

When markup.TextChanged is called you need to check if any line in the supplied range must be repainted, and send invalidate if so.

For testing you can invalide the entire screen, but for real only the minimum needed should be done. people run on laptops, and every action costs battery.


So to invalidate you need to know what you painted last, and what you paint next.
You need some structure that can store this, yet wont need to much memory.

maybe a TList of all lines.
  record
    x, y, color, heigth
  end

There may be other/better ways


Then in TextChanged you do what you currently do in GetMArkup..... and compare, invalidate, store the result.

Thinks outside the supplied line range have not changed, but if either begin or end are in the changed lines ....


--------------
More later



Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on December 04, 2015, 03:51:34 am
coming to think of it.
TextChanged means you have no indication of which lines are visible.

Imagine a file with 300.000 lines (yes such files exist) and 100.000 lines folded in the range you need to scan.

You can use SynEdit.LinesInWindow and then

Code: Pascal  [Select][+][-]
  1.   for i := 1 to SynEdit.LinesInWindow + 1 do begin
  2.     l := SynEdit.RowToScreenRow(i);
  3.     // do work for l, if l was in changed range
  4.  
  5.  
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: x2nie on December 04, 2015, 12:00:20 pm


But I found some bug made by me. Such as when a "begin" moved to the left (by deleting trailing space).
I will fix them later.
-----------------------------
You need to add invalidation calls.

Add or remove space in front of a "begin" (then resize the editor to invalidate)


I have bugfixed it in my SynEditMarkupFoldColors.pas . please update your download.
It also more simple than your suggetion, it doesn't require any list, nor checking what should be next paint.
Thanks for your LinesInWindow.   It was difficult for me to guess what is proc/prop name I need.


Anyway,
I think it would be better when the bugfix happens in TSynCustomFoldHighlighter (SynEditHighlighterFoldBase.pas),
Because, the job (invalidate rest line) has already been done (without my bugfix)
when I delete "b" from "begin",  Also, when I edit again the "egin" into "begin".


-------------
The only problem is the nested begin~end of procedure has similar color to begin~end of parent procedure.
It even annoying when the children procedure in several depth.


Try below code (copy + paste) into the Pascal Tab (or see attachment)



Code: Pascal  [Select][+][-]
  1.  
  2. procedure TSynEditMarkupFoldColors.DoTextChanged(StartLine, EndLine,
  3.   ACountDiff: Integer);
  4.  
  5.  
  6. var
  7.     y,i,LCnt : integer;
  8.  
  9.  
  10.   function GetPairCloseFold(aRow, X : integer  ): Integer;
  11.   var
  12.     y,i,LCnt : integer;
  13.     HL: TSynCustomFoldHighlighter;
  14.     NodeList: TLazSynFoldNodeInfoList;
  15.     TmpNode, CloseNode: TSynFoldNodeInfo;
  16.  
  17.  
  18.     function FindEndNode(StartNode: TSynFoldNodeInfo;
  19.                        {var} YIndex, NIndex: Integer): TSynFoldNodeInfo;
  20.       function SearchLine(ALineIdx: Integer; var ANodeIdx: Integer): TSynFoldNodeInfo;
  21.       begin
  22.         NodeList.Line := ALineIdx;
  23.         repeat
  24.           inc(ANodeIdx);
  25.           Result := NodeList[ANodeIdx];
  26.         until (sfaInvalid in Result.FoldAction)
  27.            or (Result.NestLvlEnd <= StartNode.NestLvlStart);
  28.       end;
  29.  
  30.  
  31.     begin
  32.       Result := SearchLine(YIndex, NIndex);
  33.       if not (sfaInvalid in Result.FoldAction) then
  34.         exit;
  35.  
  36.  
  37.       inc(YIndex);
  38.       while (YIndex < LCnt) and
  39.             (HL.FoldBlockMinLevel(YIndex, StartNode.FoldGroup, [sfbIncludeDisabled])
  40.              > StartNode.NestLvlStart)
  41.       do
  42.         inc(YIndex);
  43.       if YIndex = LCnt then
  44.         exit;
  45.  
  46.  
  47.       NIndex := -1;
  48.       Result := SearchLine(YIndex, NIndex);
  49.  
  50.  
  51.       if (Result.LogXEnd = 0) or (sfaLastLineClose in Result.FoldAction) then
  52.         Result.FoldAction := [sfaInvalid]; // LastLine closed Node(maybe force-closed?)
  53.     end;
  54.  
  55.  
  56.   var y2,i2 : integer;
  57.   begin
  58.     Result := -1;
  59.     y := aRow -1;
  60.  
  61.  
  62.     HL := TCustomSynEdit(self.SynEdit).Highlighter as TSynCustomFoldHighlighter;
  63.     HL.CurrentLines := Lines;
  64.     LCnt := Lines.Count;
  65.     HL.FoldNodeInfo[y].ClearFilter; // only needed once, in case the line was already used
  66.  
  67.  
  68.     NodeList := HL.FoldNodeInfo[y];
  69.     NodeList.AddReference;
  70.     try
  71.       NodeList.ActionFilter := [sfaOpen];
  72.       i := 0;
  73.       repeat
  74.         TmpNode := NodeList[i];
  75.  
  76.  
  77.         if TmpNode.LogXStart < X-1 then
  78.         begin
  79.           inc(i);
  80.           continue;
  81.         end;
  82.  
  83.  
  84.         //find till valid
  85.         while (sfaInvalid in TmpNode.FoldAction) and (i < NodeList.Count) do
  86.         begin
  87.           inc(i);
  88.           TmpNode := NodeList[i];
  89.         end;
  90.         if not (sfaInvalid in TmpNode.FoldAction) then
  91.         begin
  92.           CloseNode := FindEndNode(TmpNode, y, i);
  93.           //AddHighlight(TmpNode);
  94.           Result := CloseNode.LineIndex;
  95.           exit;
  96.         end;
  97.  
  98.  
  99.         inc(i);
  100.       until i >= NodeList.Count;
  101.  
  102.  
  103.     finally
  104.       NodeList.ReleaseReference;
  105.     end;
  106.   end;
  107.  
  108.  
  109.  
  110.  
  111.   function IsFoldMoved( aRow: Integer ): integer;
  112.   var S : string;
  113.     i,n : integer;
  114.   begin
  115.     Result := -1;
  116.     n := -1;
  117.  
  118.  
  119.     S := Caret.LineText;
  120.     for i := 1 to Min(Length(S), Length(FPrevCaretText)) do
  121.     begin
  122.       if S[i] <> FPrevCaretText[i] then
  123.       begin
  124.         n := i;
  125.         break;
  126.       end;
  127.     end;
  128.  
  129.  
  130.     if n < 0 then exit;
  131.  
  132.  
  133.     Result := GetPairCloseFold(aRow, n);
  134.     if Result > 0 then
  135.     begin
  136.       with TCustomSynEdit(SynEdit) do
  137.         Result := min(Result, TopLine +LinesInWindow);// . .RowToScreenRow(i);
  138.     end;
  139.  
  140.  
  141.   end;
  142. var
  143.   EndFoldLine,LineEnd,y : integer;
  144. begin
  145.   if EndLine < 0 then exit; //already refreshed by syn
  146.  
  147.  
  148.   y := Caret.LineBytePos.y;
  149.   EndFoldLine := IsFoldMoved(y);
  150.   if EndFoldLine > 0 then
  151.   begin
  152.     InvalidateSynLines(y+1, EndFoldLine);
  153.   end;
  154.  
  155.  
  156.   FPrevCaretText := Caret.LineText;
  157.   // I found that almost anything has been repaint by the SynEdit,
  158.   // except the trailing space editing: we should repaint them here.
  159.  
  160.  
  161. end;
  162.  
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: x2nie on December 04, 2015, 01:00:27 pm
Well, This SynEdit Fold Color is only works fine with TSynHighlighterPasSyn.
To make this useful for any other SynHighlighter,
i think we need to add some nice feature to TSynCustomFoldHighlighter  8-)


such easy way to find any pair of begin~end, or C++ "{"~"}" enabled by default.


In real world, this SynMarkup can help programmer to quickly identify the wrong logic caused by any mismatch of block-sign (such as missing block "end").
It's crucial feature when we work in complex nested block code such PHP, Pascal, and XML based code.
 
Let me describe this feature by pictures below.


-----
Ideally, TSynCustomFoldHighlighter (or TLazSynEditNestedFoldsList?) should able to provides any pair of opening+closing fold.
If it works, TSynDemoHlFold ( = simplest highlighter with folding ability) will be sexy too.

Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: x2nie on December 04, 2015, 02:03:39 pm
if you have a problem with my demo app, here my blind solution:


SynEditHighlighterFoldBase.pas line #683
Code: Pascal  [Select][+][-]
  1. function TLazSynFoldNodeInfoList.Count: Integer;
  2. begin
  3.   //if not FValid then exit(-1);
  4.   if not FValid then exit(0); //x2nie
  5.  
  6.  
  7.   DoFilter(-1);
  8.   Result := FFilteredCount;
  9. end;
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: guest58172 on December 04, 2015, 02:45:36 pm
In real world, this SynMarkup can help programmer to quickly identify the wrong logic caused by any mismatch of block-sign (such as missing block "end").
It's crucial feature when we work in complex nested block code such PHP, Pascal, and XML based code.

Does it work on the indentation rather than on open/close pairs ?
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: x2nie on December 04, 2015, 03:02:09 pm
In real world, this SynMarkup can help programmer to quickly identify the wrong logic caused by any mismatch of block-sign (such as missing block "end").
It's crucial feature when we work in complex nested block code such PHP, Pascal, and XML based code.

Does it work on the indentation rather than on open/close pairs ?
Yes and No.


Yes: the colouring aspect it self: will work in any TSynCustomFoldHighlighter,
no matter indentation (such Python) or paired block programming language (such pascal, c/c++).
Because it depends on folding system, not depends on ony programming syntax.


No: since indentation programming language (such as Python) never requires any ending-block notation,
so my TSynEditMarkupFoldColors wouldn't be able to help programmer finding missing ending-block (which is not existed).

----
To make this useful for any other SynHighlighter, i think we need to add some nice feature to TSynCustomFoldHighlighter such easy way to find any pair of begin~end, or C++ "{"~"}" enabled by default.
What I refer to begin~end, or C++ "{"~"}" is the X/Y location of that (act as opening folding & closing mark)
which are only required to locate it's position of drawing vertical lines.
So, it doesn't really mean require 'begin', 'end', '{', '}', nor anything programming language specific.


----
But unfortunately, current lazarus synhighlighter for python was not inherited from TSynEditMarkupFoldColors. So it doesn't work, but its not limitation of my class.
Even worst, it doesn't yet work with anything else either, only with TSynHighlighterPas currently works. :(

I wish in near future we can reach it really works on any TSynCustomFoldHighlighter, ... .
i think it happened because of development of lazarus's pascal editor was very far & fast, compared to development progress in SynEdit it self.
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: guest58172 on December 04, 2015, 03:24:17 pm
By the way, I suppose it only works on the master branch ? I just tried to add it to Coedit (which is based on latest stable release) and TSynEdit doesn't have a MarkupManager member... :'(
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: x2nie on December 04, 2015, 03:40:49 pm

Master branch? you might rever to git.
Yes, it require the trunk svn lazarus.
I added a public property in svn/trunk
SynEdit.MarkupManager

So you can drop a SynEdit on a form, subclass your own Markupclass, and play with it.


I will support Lazarus 1.4 too !!  8-)
but, maybe not today. :-X  Happy weekend !
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: guest58172 on December 04, 2015, 04:03:51 pm
on 1.4, I've managed to add it like this (inside my TSynEdit sub class constructor):

Code: Pascal  [Select][+][-]
  1. fMarkupIndent:= TSynEditMarkupFoldColors.Create(self);
  2. TSynEditMarkupManager(MarkupMgr).AddMarkUp(fMarkupIndent, true);

but I get some range violation, line 3325 of SynEditFoldedView

Code: Pascal  [Select][+][-]
  1. SetLength(OpenIdx, FGroupCount, FFoldNodeInfoList.Count);
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Edson on December 04, 2015, 04:21:30 pm
Well, This SynEdit Fold Color is only works fine with TSynHighlighterPasSyn.
To make this useful for any other SynHighlighter,
i think we need to add some nice feature to TSynCustomFoldHighlighter  8-)

such easy way to find any pair of begin~end, or C++ "{"~"}" enabled by default.

Well, I felt sad when discovered "TSynCustomFoldHighlighter" has not complete information about the fold levels, when I was designing my highlighter: http://forum.lazarus.freepascal.org/index.php/topic,21727.msg145713.html#msg145713
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on December 04, 2015, 05:51:44 pm
It will take a while to answer all this..... so please be patience.

Quote
I think it would be better when the bugfix happens in TSynCustomFoldHighlighter (SynEditHighlighterFoldBase.pas),
Because, the job (invalidate rest line) has already been done (without my bugfix)
when I delete "b" from "begin",  Also, when I edit again the "egin" into "begin".

The HL does indeed some of the invalidation. If a begin is inserted/removed it will invalidate down to the last affected end (potentially end of file).

But: if a "begin" simply changes x pos (add a space on that line, but somewhere before the begin) then for the HL no invalidation is needed. SynEdit will invalidate just that line (sometimes 1 or 2 extra)

Both of the above are as intended.
Code: Pascal  [Select][+][-]
  1. ...
  2.    if a then Begin // foo
  3.              |
  4.              |
  5.    end
  6.  
- if you edit "a" to "val" then the horizontal line needs to indent.
- if you edit "foo", then it does not change the line.

So only your code can know if an edit in a code-line changes the vertical lines. You then need to know how many lines below need to be invalidated.
Code: Pascal  [Select][+][-]
  1. ...
  2.    if a then Begin while b do begin// foo
  3.              |                |
  4.              |                |
  5.              |       end
  6.              |
  7.              |
  8.    end
  9.  

Code: Pascal  [Select][+][-]
  1. The only problem is the nested begin~end of procedure has similar color to begin~end of parent procedure.
I look at it later.
Including disabled nodes, and you should get them in the list, but you need to do your own count. (I mentioned before, I can add more details, but later)

Quote
Well, This SynEdit Fold Color is only works fine with TSynHighlighterPasSyn.
To make this useful for any other SynHighlighter,
i think we need to add some nice feature to TSynCustomFoldHighlighter
Yep, some code needs to be factored up from pas to the base class.

Again later. sorry , busy.

For this we also need to look at foldconfig. And that will also mean you get the ability to configure which nodes to highlight and which not.

Quote
It's crucial feature
agreed, and I try to allocate as much of my time as I can.

Quote
Code: Pascal  [Select][+][-]
  1. function TLazSynFoldNodeInfoList.Count: Integer;
  2. begin
  3.   //if not FValid then exit(-1);
  4.   if not FValid then exit(0); //x2nie
I need to check if -1 is used anywhere....


-------------
I will add more asap
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on December 04, 2015, 06:06:29 pm
Quote from: martin_fr
So only your code can know if an edit in a code-line changes the vertical lines. You then need to know how many lines below need to be invalidated.

You probably do not need to store the height of any vertical line. Since any change to that will lead to invalidation by the HL or not need full redraw.

But you need to know when a vertical line moves x pos.
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: x2nie on December 04, 2015, 06:51:50 pm
You probably do not need to store the height of any vertical line. ...
But you need to know when a vertical line moves x pos.
I did it. I've able to identify when a 'begin' is moved or not, by comparing the both texts :
* text on the line/caret before editing
* text that has been updated.
It's done because:
+ when the Caret moved up or down, I store the text in the line.
+ when text changed, I can only get the text being changed, but I already have text before change that I store when caret moved vertical.




Quote
I think it would be better when the bugfix happens in TSynCustomFoldHighlighter (SynEditHighlighterFoldBase.pas),
Because, the job (invalidate rest line) has already been done (without my bugfix)
when I delete "b" from "begin",  Also, when I edit again the "egin" into "begin".

The HL does indeed some of the invalidation. If a begin is inserted/removed it will invalidate down to the last affected end (potentially end of file).

But: if a "begin" simply changes x pos (add a space on that line, but somewhere before the begin) then for the HL no invalidation is needed. SynEdit will invalidate just that line (sometimes 1 or 2 extra)

Both of the above are as intended.
if my bugfixed invalidation runs, so the rest lines possibly invalidated twices !
Can it configured outside SynMarkup? As you said, people play in laptop, and every invalidate cost battery ha ha haa... >:D




Quote
Quote
Well, This SynEdit Fold Color is only works fine with TSynHighlighterPasSyn.
To make this useful for any other SynHighlighter,
i think we need to add some nice feature to TSynCustomFoldHighlighter
Yep, some code needs to be factored up from pas to the base class.


Quote
It's crucial feature
agreed, and I try to allocate as much of my time as I can.
Thanks you!


----
But the position of 'begin'~'end' (or '{' and '}' in java/c) is not for drawing vertical purpose only.
Another reason we need it (knowing their coordinate) is also for drawing that pair in same color as vertical lines.
and in SynMarkup, drawing text in nested color require the position of those elements.


____
My bottom line is: how to make Lazarus FoldHL (Demo) multi level color folding with my SynMarkup class,
by a little effort.




----
My next plan is to draw horizontal dotted line when the left position of begin & end is horizontally different.
But, before do that, I should able to detect the deferential between begin & end, from up to bottom
I think the easiest way is when HL puts this info on StartCodeFoldBlock & EndCodeFoldBlock.

But, Maybe it couldn't be done automagically, because each HL has different way to parse, hence different possibility of when to callStartCodeFoldBlock & EndCodeFoldBlock (can't make sure if a new token has already stored or not yet). Hence, the TSynCustomFoldHighlighter couldn't make sure where the token starts nor when it ends.


So, maybe  giving the each HL a chance to tell TSynCustomFoldHighlighter of where is the opening fold & closing fold located, is perfect solution.
When the open~close token attached to a range, we can easily detect the 'end' coordinate when drawing the row of 'begin'.
 :)
Code: Pascal  [Select][+][-]
  1.  
  2. procedure TSynDemoHlFold.Next;
  3. begin
  4.   inherited Next;
  5.   if (copy(FLineText, FTokenPos, FTokenEnd - FTokenPos) = '-(-') then
  6.   begin  
  7.      StartCodeFoldBlock(nil);
  8.  
  9.      // store the opening fold bracket after push / attached new child-area
  10.      StartCodeFoldToken( FTokenPos, FTokenEnd ); // if level increased
  11.  
  12.   end;
  13.  
  14.   if (copy(FLineText, FTokenPos, FTokenEnd - FTokenPos) = '-)-') then
  15.   begin  
  16.     // put the closing fold bracket before pop / detached, back to parent work-area
  17.      EndCodeFoldToken( FTokenPos, FTokenEnd );  // level decreasing.
  18.  
  19.      EndCodeFoldBlock;(
  20.   end;
  21. end;
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on December 04, 2015, 09:38:46 pm
Quote
I think the easiest way is when HL puts this info on StartCodeFoldBlock & EndCodeFoldBlock.
Yes that code needs to move up to the base class.

The important things (and just from top of my head, there may be more).

FoldInfo is only collected if FCatchNodeInfo is true.
That is when TSynPasSyn.InitFoldNodeInfo is called.

This affects SetLine !!!!! And I need to look into detail why, but it will be important.
So if FCatchNodeInfo  moves up, it must have a protected readable property.

InitFoldNodeInfo must also move up, but parts are pas specific.

FOLDGROUP_* is only pascal (it is for ifdef folds). Other HL may later get that too, but different issue / later.

It isnt going to be easy to move, because some of the flags sfaOpen ... are pas specific.

Some come from fold config. Or depend on it.

For any HL to use this, they need fold config.

Also foldconfig should get a new flag sfaMarkupOutline, that will enable/disable outlining (your markup would check for the presence of it)

There may be a 2nd flag needed to help changing colors on nested procs (you still need to do counting in the markup, you can not use the levels directly)

---------------------------
I get more on it later.


You could start by adding a copy to your demo hl, then get the understanding, and that will later help deciding on how to refactor
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on December 04, 2015, 09:42:41 pm
DoCaretChanged for invalidate is not enough.

text can change by codetool and other mean (2 shared editors, text edited by one changes in both). then the caret does not move.
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on December 08, 2015, 07:16:45 am
OK, 2 bits of info.

1) foldconfig.

currently length is set in .Create. not good, maybe move.

GetFoldConfigInstance
gets the initial setting.

for each foldtype (cfbtBeginEnd,...) it indicates, if it can
Code: Pascal  [Select][+][-]
  1. TSynCustomFoldConfigMode = (fmFold, fmHide, fmMarkup);
- fold
- hide
- markup (word triplet at caret)

you could add fmMarkupOutline then this can be configured, and the HL can indicate which nodes are for your module.

the HL can set 2 flags (maybe more)
- sfaMarkupOutline
  visible outline
- sfaMarkupOutlineIndend
  not outlined, but increase color (e.g. a nested procedure block)

those would then be set in HL.InitNode() according to the language rules.

As for the "level" of intend. The HL levels are not correct, but when you go through Nest : TLazSynEditNestedFoldsList; you could keep count of nodes that have the flag (you need to start left to right then).

Or simply set them on the result FHighlights. But then you need to add dummy entries for the invisible sfaMarkupOutlineIndend. So that is not desirable.

-------------------------------
As for adding fold info to the base class.

PasHl has a lot of custom code. I wouldnt move it just yet.
Add virtual methods to the base class with whatever generic code is possible, and keep them overriden in pasHl

Then you can start getting code for the other HL.

It is important to collect node info only when requested. (per line). So it does not slow down normal hl scans.
 
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: x2nie on December 08, 2015, 07:55:44 am
PasHl has a lot of custom code. I wouldnt move it just yet.
Add virtual methods to the base class with whatever generic code is possible, ....

It is important to collect node info only when requested. (per line). So it does not slow down normal hl scans.

Then you can start getting code for the other HL.
All above is done. I have created TSynColorFoldHighlighter that can be used by other HL as base class for quick creating   HL with folding+ nested color ability. Yes, it harvest the node info only when requested, so it is efficient.


For demo, I created TSynHighlighterBracket = class (TSynColorFoldHighlighter)
it works gracefully. (see attachment)

I didn't synchro the the pasHL yet, so it still:
Code: Pascal  [Select][+][-]
  1. TSynPasSyn               = class(TSynCustomFoldHighlighter)
  2. TSynColorFoldHighlighter = class(TSynCustomFoldHighlighter)
  3. TSynHighlighterBracket   = class(TSynColorFoldHighlighter)
  4.  


-----
Yesterday and before, I experimented with adding properties directly to TSynCustomHighlighterRange.
It was more efficient because doesn't need to re-scan, no recalculation needed.
It also been worked, but it was unstable somehow. When I scroll the SynEdit, colors changed unpredictably.
Days and night curiosity not paid. It was nightmare for these days.  >:D
I ignored those all changes and create a whole new middle class "TSynColorFoldHighlighter" and today was beautiful day.  8-)


-----
Now the problem is TLazSynEditNestedFoldsList doesn't work at all. It is a bit mysterious  for my brain.
Any clue ?


-----
(I am studying your last post.)
.
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on December 08, 2015, 06:31:33 pm
All above is done. I have created TSynColorFoldHighlighter that can be used by other HL as base class for quick creating   HL with folding+ nested color ability. Yes, it harvest the node info only when requested, so it is efficient.

I look at this later. The aim should be to patch the TSynCustomFoldHighlighter.

Quote
Yesterday and before, I experimented with adding properties directly to TSynCustomHighlighterRange.
It was more efficient because doesn't need to re-scan, no recalculation needed.
Not sure what you did, but the range (and all data/classes held by it) must be immutable.

The exception is the one used to work on the current line.
See SetRange(), GetRange() they make copies/assign.
New fields must be in the compare method.

But new fields should be kept to a minimum. I dont think the color level is needed. It is easy enough to calculate.

I just started wondering if the existing FPasFold...Level are really needed. But that is not part of this issue here.

Quote
Now the problem is TLazSynEditNestedFoldsList doesn't work at all. It is a bit mysterious  for my brain.
Any clue ?

what exactly goes wrong?


One way would be to look at SourceSynEditor, and copy TIDESynGutterDebugHL
It lets you spy on your data (modify it / edit the Paint() to show what you need.), then you can see if all values in the HL are as expected


for the bracket fold hl , I get an assert
TLazSynEditNestedFoldsList.InitNestInfoForIndex
Code: Pascal  [Select][+][-]
  1.   assert(nfeHasHNode in FNestInfo[AnIndex].FFLags, 'nfeHasHNode in FNestInfo[AnIndex].FFLags');
  2.  

looks like it fails here
Code: Pascal  [Select][+][-]
  1.       if l >= GrpCnt[t] then continue;
  2.  

maybe AHighlighter.FoldBlockEndLevel()
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: x2nie on December 09, 2015, 06:40:27 am
.. TSynColorFoldHighlighter  can be used by other HL as base class
for quick creating   HL with folding+ nested color ability. Yes, it harvest the node info only when requested, so it is efficient.
.. The aim should be to patch the TSynCustomFoldHighlighter.
Owh? okay, I think that is good idea. 8)
it would be someday, when the TSynPasSyn was redesigned too, to reflect that changes.
But now, while lazarus deeply requires the stable TsynPasSyn, I can not do that.


Quote
Now the problem is TLazSynEditNestedFoldsList doesn't work at all. It is a bit mysterious  for my brain.
Any clue ?
what exactly goes wrong?
The vertical lines (generated by nested fold info) aren't painted. I guess it will be solved by following your (last post) guidance.


I still trying
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on December 09, 2015, 07:06:26 am

The vertical lines (generated by nested fold info) aren't painted. I guess it will be solved by following your (last post) guidance.

I didnt debug it to the end, but I noticed that a loop in TLazSynEditNestedFoldsList.InitNestInfoForIndex( exited early
Quote
looks like it fails here
Code: Pascal  [Select][+][-]
  1.       if l >= GrpCnt[t] then continue;
  2.  

maybe AHighlighter.FoldBlockEndLevel()

I then get the assert
Code: Pascal  [Select][+][-]
  1.   assert(AnIndex >= FEvaluationIndex, 'TLazSynEditNestedFoldsList.InitNestInfoForIndex Index not found');

A bit of background (you need to know fold levels, see wiki, but I guess you know them by now

TLazSynEditNestedFoldsList looks for nodes that opened in a previous line AND are still open in the requested line.

GrpCnt[t] is the foldlevel from the requested line. (numbers may be off by one, if it is end vs start of fold.
Code: Pascal  [Select][+][-]
  1. begin  // level 0
  2.   begin    // level 1 - not of interest, will close before requested line
  3.   end  
  4.   // requested line/ level 1
  5. end
  6.  
the example is not real, it would actually require several folds starting on the same line, but ignore....


when debuginh the  "-(-"  HL, GrpCnt[t] is always 0, that is wrong, it is set in InitSubGroupEndLevels

so I guess AHighlighter.FoldBlockEndLevel()  is not working.

--------------
GrpCnt = GroupCount , count for group ....

ingnore group, except for pas there is only one, with index 0 (zero).

in pascal there are 2 more groups for IFDEF and %REGION, because they can overlap with begin/end and each other, so they need there own count.

Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on December 09, 2015, 07:43:09 am
FoldBlockEndLevel may not know about the "pas" level, that you copied from the pas hl.

so you need to copy more code.

on the other hand, I suggest: drop the pas level again.

the levels in the HL are not always correct for your purpose of color picking. and even if they are now, they may change....

see my earlier mail about counting while going through the list.
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: x2nie on December 09, 2015, 09:21:47 am
 :'( I am now completely confused. (but not frustrated).  :D  I don't really know what I have copied from SynPas  :-[ .




Yeah, Good advice, I think I need to restart from the very beginning.
For honest, actually I did only read a few of your earlier posts. I will read them all by now on.


-----
Anyway, as suggested, I've copy the debug gutter, so I can see the visible levels for each line.
(see attachmeng) Thanks you.
Therefor,  I am sure I have missed something in the middle work.


I will back when get something else.
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: x2nie on December 09, 2015, 03:42:14 pm

I have read wiki and many topic at this forum related to folding-synedit.

Good news: the nested-level is now back on track. Everything is good.
I guess, The bug was wrong value of FoldGroup:
Code: Pascal  [Select][+][-]
  1. node.FoldGroup := 0; // <--- wrong. the bug.
  2. node.FoldGroup := 1;//FOLDGROUP_PASCAL; //correct somehow


---
when debuginh the  "-(-"  HL, GrpCnt[t] is always 0, that is wrong, it is set in InitSubGroupEndLevels
so I guess AHighlighter.FoldBlockEndLevel()  is not working.
It is now too complicated for me to support both old and new classes.
I will drop the FoldHL.pas soon, I have drop it, because and SynHighlighterBracket is intended as replacement of that demo.


on the other hand, I suggest: drop the pas level again.

the levels in the HL are not always correct for your purpose of color picking. and even if they are now, they may change....
I am ready now. What is exactly the idea of "drop the pas level"?
does it means erasing below prop/proc from my TSynColorFoldHighlighterRange:
Code: Pascal  [Select][+][-]
  1.  
  2.     property PasFoldEndLevel: Smallint read FPasFoldEndLevel write FPasFoldEndLevel;
  3.     property PasFoldFixLevel: Smallint read FPasFoldFixLevel write FPasFoldFixLevel;
  4.     property PasFoldMinLevel: Smallint read FPasFoldMinLevel write FPasFoldMinLevel;
  5.  
  6.     function Add(ABlockType: Pointer = nil; IncreaseLevel: Boolean = True):
  7.         TSynCustomCodeFoldBlock; override;
  8.     procedure Pop(DecreaseLevel: Boolean = True); override;
  9.     procedure Clear; override;
  10.     function Compare(Range: TSynCustomHighlighterRange): integer; override;
  11.     procedure Assign(Src: TSynCustomHighlighterRange); override;
?
Above properties + methods is the heart of my nested-color-folding. without them, the color of opening tag cant be equal to the color of closing tag.


Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: x2nie on December 09, 2015, 04:14:42 pm
see my earlier mail about counting while going through the list.
I didn't understand of what and how of that counting...  . Would you like to be more specific?
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: x2nie on December 09, 2015, 04:26:24 pm
Well, This SynEdit Fold Color is only works fine with TSynHighlighterPasSyn.
To make this useful for any other SynHighlighter,
i think we need to add some nice feature to TSynCustomFoldHighlighter  8-)

such easy way to find any pair of begin~end, or C++ "{"~"}" enabled by default.

Well, I felt sad when discovered "TSynCustomFoldHighlighter" has not complete information about the fold levels, when I was designing my highlighter: http://forum.lazarus.freepascal.org/index.php/topic,21727.msg145713.html#msg145713 (http://forum.lazarus.freepascal.org/index.php/topic,21727.msg145713.html#msg145713)


@Edson, can we continue discussing that here? if you still want to design a folding-highliter.
Fold Level SynHL was very hard to create, I am sure because I have tried it my self.
but it seem to be (nearly) easier nowadays.
What capabilities was your expected HL?


@A-M (the topic's author), hey you are disappearing when I were coming. it seem that I is you ha ha ha... No,
we are different person. what SynEditHighlighter you need? only pascal, eh?
 
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Edson on December 09, 2015, 05:30:43 pm
@Edson, can we continue discussing that here? if you still want to design a folding-highliter.
Fold Level SynHL was very hard to create, I am sure because I have tried it my self.
but it seem to be (nearly) easier nowadays.
What capabilities was your expected HL?

Hi @x2nie. I have already designed a folding-highlighter: https://github.com/t-edson/SynFacilSyn
In it, the support for folding is not as good as I wanted, mainly because of some limitation I found in TSynCustomFoldHighlighter.
Summarizing, I wanted this class understood about block of syntax, but it only knows about showing folding marks on the gutter > >:(.
I think this limitation can affect you too, on your development, because I have the feeling (no time to analyze all the posts  :-X) the feature you are implementing need to know about the syntax of the language.

Maybe it's better open a new thread "Modernizing the TSynCustomFoldHighlighter class"  :-\.
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on December 09, 2015, 05:48:32 pm
Hi @x2nie. I have already designed a folding-highlighter: https://github.com/t-edson/SynFacilSyn
In it, the support for folding is not as good as I wanted, mainly because of some limitation I found in TSynCustomFoldHighlighter.
Summarizing, I wanted this class understood about block of syntax, but it only knows about showing folding marks on the gutter > >:(.

I am not exactly sure what you try to describe?

But the HL only provides what is foldable. It does not provide the GUI to fold it.
1) It is possible to fold code by clicking on the text (e.g. "begin"). Use advanced mouse settings.
2) you can write alternative gutter part code
3) you can write a markup to colorize fold keywords.

what you currently can not do, is to draw a [ + ] or other sign, in the middle of the text.
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Edson on December 09, 2015, 06:24:05 pm
Hi Martin,

I didn't worry, and I don't worry about the way the user interact with the folding or how it's shown in the gutter.

I was talking about some limitations (Maybe "features not implemented" sounds better) of TSynCustomFoldHighlighter:

1: There is not way to obtain the total of blocks opened (IIRC FoldIndex only count the opened with folding mark). I know Pascal HL do  :D.
2. There is not a way to know if a block was opened and closed at the same line. I know Pascal HL do  :-\.
3. (This is a new feature I wish) I would like to have some facilities to close a block before a token starts (http://forum.lazarus.freepascal.org/index.php/topic,23411.msg139608.html#msg139608). I know Pascal HL do  :o.

Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: x2nie on December 10, 2015, 03:54:54 am
I wanted this class understood about block of syntax, but it only knows about showing folding marks on the gutter .
I am not exactly sure what you try to describe?
... the HL only provides what is foldable. It does not provide the GUI to fold it.
I felt understood what Edson refer to.
That is, TSynCustomFoldHighlighter ("the HL") provides which "block" is foldable: by searching the fold level for each line.
Therefore, knowing the LineIndex is enough for TSynCustomFoldHighlighter it self to make foldable feature works.

But, for ancestor (like Edson own), it seem not enough. Because it also need to know the column index too for their internal work (such differentiate coloring per block-fold according to it's fold-dept).
(and another problem begun here!  :P )


Simply, the difference is about how to define of a "block".
Let make it simple, Suppose this code:
Code: Pascal  [Select][+][-]
  1. procedure Foo();
  2. begin
  3.     if Bar() then
  4.     begin                         // "inner-block" start
  5.         var_bean := const_nut;    // "inner block" inside
  6.         var_coffee := const_nut;    // "inner block" inside
  7.    end                           // "inner-block"  finish
  8.     else
  9.        var_bean := const_jar;
  10. end;
  11.  
  12. 0        1         2
  13. 12345678901234567890

All with excluding comment-statements will be :
The new problem begun here! which problem?
when 'begin' is mixed with other block:
Code: Pascal  [Select][+][-]
  1. procedure Foo();
  2. begin
  3.     if Bar() then begin                      // "inner-block" start
  4.         var_bean := const_nut;    // "inner block" inside
  5.         var_coffee := const_nut;    // "inner block" inside
  6.     end                           // "inner-block"  finish
  7.     else       var_bean := const_jar;
  8. end;
  9. 0        1         2
  10. 12345678901234567890
.. the sum of combination of how to define the block exploded! TSynCustomFoldHighlighter souldn't care about it, right?

Summary:
Feel contradiction, eh?
I think the compromised approach is to make TSynCustomFoldHighlighter holds everything, except the fold-block-definition. Wow, sexy right?
It is sexy because no matter columns definition, the TSynCustomFoldHighlighter it self doesn't need it, wouldn't change it, wouldn't be disturbed.
Meaning: we can  reuse all things implemented in current version, while those new fields added.
The ancestor (such TmySynLFMSyn2) is the who only take care about what data stored there and how to make them useful.


Enough of theory, give me the code (Linus said).
Code: Pascal  [Select][+][-]
  1. function TSynLFMSyn.StartLfmCodeFoldBlock(ABlockType: TLfmCodeFoldBlockType): TSynCustomCodeFoldBlock;
  2. var
  3.   FoldBlock: Boolean;
  4.   p: PtrInt;
  5. begin
  6.   FoldBlock :=  FFoldConfig[ord(ABlockType)].Enabled;
  7.   p := 0;
  8.   if not FoldBlock then
  9.     p := PtrInt(CountLfmCodeFoldBlockOffset);
  10.   //Result := StartCodeFoldBlock(p + Pointer(PtrInt(ABlockType)), FoldBlock);
  11.   Result := StartCodeFoldBlock(FTokenPos, Run, p + Pointer(PtrInt(ABlockType)), FoldBlock);
  12. end;
  13.  
  14.  
  15. procedure TSynLFMSyn.EndLfmCodeFoldBlock;
  16. var
  17.   DecreaseLevel: Boolean;
  18. begin
  19.   DecreaseLevel := TopCodeFoldBlockType < CountLfmCodeFoldBlockOffset;
  20.   //EndCodeFoldBlock(DecreaseLevel);
  21.   EndCodeFoldBlock(FTokenPos, Run, DecreaseLevel);
  22. end;
This is, the ancestor is given a chance by TSynCustomFoldHighlighter to tell the X and X2 (columns).
The Y (LineIndex) is already know by TSynCustomFoldHighlighter, so no need to ask as parameter.
But how to make X and X2 usefull, is completely up to ancestor since TSynCustomFoldHighlighter may ignore the columns value.


---
Conclusion:
The absent of columns info on TSynCustomFoldHighlighter is not a bug, nor it is "feature not implemented"
IMHO it was rather intended by design. see the above problem.
But, once we can generalized the need as "X1,X2,Y" values, we can add this new feature.
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: x2nie on December 10, 2015, 05:35:21 am
...I have the feeling..,  the feature you are implementing need to know about the syntax of the language.
No, it wouldn't.


1. : I have satisfied with current progress of my coloring SynMarkup, which is the keywords doesn't colored. only 'begin' and 'end' is colored.
Anyway, The Markup doesn't scan/parse any programming syntax.
Nor it require of such knowledge of any specific programming language syntax to do markup (coloring)'s job.
Therefore, if/then/else/for/while/case/... is not colored here. But I am happy with this, since the keywords was already black+bold.


2. : Once I really need to color the pascal keyword according to depth of pascal statements,
 the synMarkup will only need to ask to HL about: at where coordinate should be painted with what color.
Hence "what color" is assumed by the Markup it self, HL only told some info such : its a opening-fold, it's the 2nd level of fold-dept, etc.

Quote
Maybe it's better open a new thread "Modernizing the TSynCustomFoldHighlighter class"  :-\ .
Yeah, good idea.
I am in beginning of collecting as much as possible ideas + many of conflicting requirement of new fold HL.
When it done, I will provide a patch / open new topic.
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on December 10, 2015, 05:45:35 am
very quick note / I go into more details (and all the other questions) later.
----------------------------

about the x position.

It is intentionally NOT stored. not even in the pas HL. The idea is to store less data. (yet each HL can decide to store added info, though the range is not a good place for it, as each range should by design apply to as many lines of code as possible, and should be re-used).

X can be calculated. Each HL can implement the same as the pas hl.

the procedure for getting the calculated x is in the TSynCustomFoldHighlighter , so if a HL overrides it, it will work.

Adding common code to TSynCustomFoldHighlighter, so it needn't be copied into each hl can be done, but it is not a show stopper.
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: x2nie on December 10, 2015, 05:50:29 pm
Furthest progress: I have merged my middleware-HL into TSynCustomFoldHighlighter.


Known issue:
- FFoldConfig aren't yet supported. I don't have have idea when user gives nil as ABlockType.
- SingleLine still appears as foldable everywhere, but when is clicked the folded area is wrong (see LFM tab)
- TSynPasSyn is untouched.


- since pascal HL is kept unmodified, I keep both old way and the new way,
 the side effect of this keeping two gate, the old way call shall implement it's own NestFoldLevels.
For example:  FoldHL.pas (shipped with lazarus) doesn't work; see my FoldHL.pas for what is difference.
I reenabled the Fold tab, but now the error occured on XML tab. I wouldn't resolve it.
My priority is now going to pascal HL.

Code: Pascal  [Select][+][-]
  1. // Open/Close Folds
  2. //old gate:
  3. function StartCodeFoldBlock(ABlockType: Pointer;
  4.         IncreaseLevel: Boolean = true): TSynCustomCodeFoldBlock; virtual; overload;
  5. procedure EndCodeFoldBlock(DecreaseLevel: Boolean = True); virtual;
  6.  
  7.  
  8. //new gate:
  9. function StartCodeFoldBlock(LogX1,LogX2 : Integer; ABlockType: Pointer=nil;
  10.         IncreaseLevel: Boolean = true): TSynCustomCodeFoldBlock; virtual; overload;
  11. procedure EndCodeFoldBlock(LogX1,LogX2 : Integer;
  12.         DecreaseLevel: Boolean = True); virtual; overload;  
I think we should drop the old gate, and only accept the new one ?


-----

The code can be downloaded:
https://github.com/x2nie/syneditmarkupnestedcolors/blob/master/SynEdit-modified/synedithighlighterfoldbase.pas (https://github.com/x2nie/syneditmarkupnestedcolors/blob/master/SynEdit-modified/synedithighlighterfoldbase.pas)


svn: https://github.com/x2nie/syneditmarkupnestedcolors (https://github.com/x2nie/syneditmarkupnestedcolors)
git: https://github.com/x2nie/syneditmarkupnestedcolors (https://github.com/x2nie/syneditmarkupnestedcolors)
zip: https://github.com/x2nie/syneditmarkupnestedcolors/archive/master.zip (https://github.com/x2nie/syneditmarkupnestedcolors/archive/master.zip)

Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on December 10, 2015, 10:36:19 pm
I was talking about some limitations (Maybe "features not implemented" sounds better) of TSynCustomFoldHighlighter:

1: There is not way to obtain the total of blocks opened (IIRC FoldIndex only count the opened with folding mark). I know Pascal HL do  :D.
You can add tracking them like pas-hl, but as I said maybe pas-hl can even get rid of it. Yet even then your hl can add it. It is not a feature that all HL need, so it still is open if or where to add it
Or simply walk up parent links and count.
Quote
2. There is not a way to know if a block was opened and closed at the same line. I know Pascal HL do  :-\.
then there is a way, isn't there?
If you are closing a fold, and after closing you are greater or equal to the MinFoldLevel, then it was opened on the same line.
MinFoldLevel must be maintained (after each of the above compare):
- it starts (at the start of line) with the current fold level.
- if you close a block and you get below it, then lower it too
That is assuming you process from left to right.
Quote
3. (This is a new feature I wish) I would like to have some facilities to close a block before a token starts (http://forum.lazarus.freepascal.org/index.php/topic,23411.msg139608.html#msg139608). I know Pascal HL do  :o.
like the "var" folding?
Code: Pascal  [Select][+][-]
  1. procedure a;
  2. var
  3.   i: boolean;
  4. begin
The "begin" closes the var block. But it is not part of it.
Search LastLinePasFoldLevelFix. But you can only go one line back (if there are empty lines they must be in the fold.
The importance is, that if a line (e.g. line 30) changes, the HL needs to tell the editor that the line above changed. See the comment in
  procedure TSynCustomHighlighter.ScanRanges;
that needs to be made optional.


Therefore, knowing the LineIndex is enough for TSynCustomFoldHighlighter it self to make foldable feature works.
But, for ancestor (like Edson own), it seem not enough. Because it also need to know the column index too for their internal work (such differentiate coloring per block-fold according to it's fold-dept).
(and then example code....)

This is about moving the CatchNodeInfo into the TSynCustomFoldHighlighter?

Because all x pos can be gotten with CatchNodeInfo, only currently each HL needs to implement from scratch.
Quote
Code: Pascal  [Select][+][-]
  1.   Result := StartCodeFoldBlock(FTokenPos, Run, p + Pointer(PtrInt(ABlockType)), FoldBlock);
Do not pass the data as param. It is only needed if CatchNodeInfo is on.

Define a virtual method that gets the boundaries, override it each HL (just a few lines of code), and TSynCustomFoldHighlighter can call it and get the info.


- SingleLine still appears as foldable everywhere, but when is clicked the folded area is wrong (see LFM tab)
still happening?
Quote
For example:  FoldHL.pas (shipped with lazarus) doesn't work; see my FoldHL.pas for what is difference.
ups then it needs to be fixed. Do you have a patch.
Please report a bug. (with or without patch)
Quote
Code: Pascal  [Select][+][-]
  1. //new gate:
  2. function StartCodeFoldBlock(LogX1,LogX2 : Integer; ABlockType: Pointer=nil;
  3.         IncreaseLevel: Boolean = true): TSynCustomCodeFoldBlock; virtual; overload;
  4. procedure EndCodeFoldBlock(LogX1,LogX2 : Integer;
  5.         DecreaseLevel: Boolean = True); virtual; overload;  
I think we should drop the old gate, and only accept the new one ?
see comment above. Keep old interface, and define a getter.
Quote
The code can be downloaded:
https://github.com/x2nie/syneditmarkupnestedcolors/blob/master/SynEdit-modified/synedithighlighterfoldbase.pas (https://github.com/x2nie/syneditmarkupnestedcolors/blob/master/SynEdit-modified/synedithighlighterfoldbase.pas)
Eventually a patch will be needed.

But for now you can keep it there.
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: x2nie on December 11, 2015, 03:58:23 am
Therefore, knowing the LineIndex is enough for TSynCustomFoldHighlighter it self to make foldable feature works.
But, for ancestor (like Edson own), it seem not enough. Because it also need to know the column index too for their internal work (such differentiate coloring per block-fold according to it's fold-dept).
(and then example code....)

This is about moving the CatchNodeInfo into the TSynCustomFoldHighlighter?

Because all x pos can be gotten with CatchNodeInfo, only currently each HL needs to implement from scratch.
I think No, its not about the normal way of how to get x pos.
(normal = sample code = finding X,Y as such in TSynEditMarkupWordGroup.FindMatchingWords() ).


Rather, there is a need to find the pair of opening~closing fold tag in simplest way!
You know, in FindMatchingWords: FindEndNode / FindStartNode methods are too complicated for such this easy task: get opening~closing fold tag.
(simplest = storing that (opening+closing fold tag) info into whether a CodeFoldBlock or into each FoldRange?).


So, normal way is bad (or at least : not optimized) because this way is finding through too many lines (assumed 10000 lines) until it match.
Storing that info will save time and battery. :)p
Code: Pascal  [Select][+][-]
  1.   function FindEndNode(StartNode: TSynFoldNodeInfo;
  2.                        var YIndex, NIndex: Integer): TSynFoldNodeInfo;
  3.     function SearchLine(ALineIdx: Integer; var ANodeIdx: Integer): TSynFoldNodeInfo;
  4.     begin
  5.       NodeList.Line := ALineIdx;
  6.       repeat
  7.         inc(ANodeIdx);
  8.         Result := NodeList[ANodeIdx];
  9.       until (sfaInvalid in Result.FoldAction)
  10.          or (Result.NestLvlEnd <= StartNode.NestLvlStart);
  11.     end;
  12.   begin
  13.     Result := SearchLine(YIndex, NIndex);
  14.     if not (sfaInvalid in Result.FoldAction) then
  15.       exit;
  16.  
  17.  
  18.     inc(YIndex);
  19.     while (YIndex < LCnt) and
  20.           (HL.FoldBlockMinLevel(YIndex, StartNode.FoldGroup, [sfbIncludeDisabled])
  21.            > StartNode.NestLvlStart)
  22.     do
  23.       inc(YIndex);
  24.     if YIndex = LCnt then
  25.       exit;
  26.  
  27.  
  28.     NIndex := -1;
  29.     Result := SearchLine(YIndex, NIndex);
  30.  
  31.  
  32.     if (Result.LogXEnd = 0) or (sfaLastLineClose in Result.FoldAction) then
  33.       Result.FoldAction := [sfaInvalid]; // LastLine closed Node(maybe force-closed?)
  34.   end;      
The very bad code in #19..23
Even if there was no alternative way, we still need to do that searching/calculation easier, by such wrap it in a function maybe enough.


Quote
- SingleLine still appears as foldable everywhere, but when is clicked the folded area is wrong (see LFM tab)
still happening?
Quote
For example:  FoldHL.pas (shipped with lazarus) doesn't work; see my FoldHL.pas for what is difference.
ups, then it needs to be fixed. Do you have a patch.
Please report a bug. (with or without patch)
Sorry that is my fault. It's not happen in original/lazarus SynLFM. It occured only in my SynLFM2.pas


Quote
Keep old interface, and define a getter.
Nice idea!


Quote
Eventually a patch will be needed.
I forgot how to create a Svn patch?
But you can see all changes separated from original code here: https://github.com/x2nie/syneditmarkupnestedcolors/commit/2438a0298c73942e6545bc29be1bbeb194db4c7c
Or when you are visiting github, click the "38 commits', click a commit node (most top) and there will be file per file, changes are green.
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on December 11, 2015, 04:46:25 am
I think No, its not about the normal way of how to get x pos.
(normal = sample code = finding X,Y as such in TSynEditMarkupWordGroup.FindMatchingWords() ).

Rather, there is a need to find the pair of opening~closing fold tag in simplest way!
You know, in FindMatchingWords: FindEndNode / FindStartNode methods are too complicated for such this easy task: get opening~closing fold tag.
(simplest = storing that (opening+closing fold tag) info into whether a CodeFoldBlock or into each FoldRange?).

So, normal way is bad (or at least : not optimized) because this way is finding through too many lines (assumed 10000 lines) until it match.
Storing that info will save time and battery. :)p
Quote
The very bad code in #19..23
Even if there was no alternative way, we still need to do that searching/calculation easier, by such wrap it in a function maybe enough.
now we have to differentiate several things
1) the data itself (what is stored and how)
2) How to get other info from it, and what methods are provided.

(1)
I still think foldlevel is the correct data to store.

I see no need to store for example that a fold starting in line 190 has a length of 75 lines. (store 75)
a) you know the length only when you hit line 265, then you have to backtrack the same way as current search (just backward). Only in the HL you do so for every line, even if not visible / not needed.
b) you need extra storage, and since you can have 100 fold opening on the same line, you need more often to allocate/deallocate memory (not very fast).
c) if you store it in the range, or code foldblock (part of the range) then they become nearly impossible to re-use.
Example pascal: I tried loading several 100k lines (in many units) of code. There will only be about 1700 range objects. (Yes, you can write code that forces more, way more.)

One could (but later) discuss a cache for lines in the visible area (that would not be part of the HL)

(2)
TSynCustomFoldHighlighter.FoldEndLine ?

But it should be added to TLazSynEditNestedFoldsList, which could not only return the line but also the info node. And which, if nested folds are scanned can continue scanning from the last known line. And do other small caching tasks.

As for long folds (10000 lines), that is no problem. Well if you scan a 1000 folds of 10k lines then maybe it will be.
A simple loop with just one compare (as abort condition) can easily loop over a million lines.
Having said that: folding 100k lines takes about 1 to 2 sec on my 5 year old pc. But it might be something else that takes the time. Because the scrollbar is adjusted almost immediately, and that can only happen when it has the line count. So I think something else takes the time.

Quote
I forgot how to create a Svn patch?
But you can see all changes separated from original code here: https://github.com/x2nie/syneditmarkupnestedcolors/commit/2438a0298c73942e6545bc29be1bbeb194db4c7c
Or when you are visiting github, click the "38 commits', click a commit node (most top) and there will be file per file, changes are green.
We can create it from git.
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: x2nie on December 11, 2015, 10:06:48 am
I have good news and a question.


* a new virtual method has been introduced for getting X,X2 location of fold mark
Code: Pascal  [Select][+][-]
  1. procedure TSynCustomFoldHighlighter.GetTokenBounds(out LogX1, LogX2,
  2.   LogVertGuideX: Integer); //Vert Guide Line, used by nested color markup
  3. var p : pchar; L : integer;
  4. begin
  5.   GetTokenEx(p,L);
  6.   LogX1 := GetTokenPos;
  7.   LogX2 := LogX1 + L -1;
  8.   LogVertGuideX := LogX1;
  9. end;
By this, almost non of HL need to override, except the XML HL (see attachment)
Surprisingly, all of HL works like charms (fine).


But...
the MarkupWordGroup seem as wrong on Node.LogXEnd, mean conflict with my idea.
See the "food"~"/food"  vs  "repeat"~"until" screengrab.


* My idea is   : Node.LogXEnd = the last char.
* while MWG : Node.LogXEnd = the last char +1
In other word:
Code: Pascal  [Select][+][-]
  1.   object
  2. 012345678901234567890
  3. 0          1         2
above "object" keyword has Node values of
* My LogXEnd= 7, LogXStart=2
* MWG LogXEnd= 8, LogXStart=2


So, my question is: which one to keep? mine our MWG? by what reason?
My personal reason is in pascal, the width of string is the last char. NOt the width + 1
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: x2nie on December 11, 2015, 10:33:26 am
please ignore my previous question. it was stupid question :-[
I have resolved it. of course reusing the known code (MarkupWordGroup) is the winner.
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: x2nie on December 11, 2015, 11:06:21 am

@martin,

Now, the problem is when text/line trailing with tab. The vertical lines drawn improperly.
I have no idea, what's problem? I have no clue :'(
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: x2nie on December 11, 2015, 02:38:34 pm
So, normal way is bad (or at least : not optimized) because this way is finding through too many lines (assumed 10000 lines) until it match.
Storing that info will save time and battery. :)p
...

I see no need to store for example that a fold starting in line 190 has a length of 75 lines. (store 75)

c) if you store it in the range, or code foldblock (part of the range) then they become nearly impossible to re-use.

As for long folds (10000 lines), that is no problem. Well if you scan a 1000 folds of 10k lines then maybe it will be.
Hey, I currently face a real problem ... >:D  !


coming is new bad news (and the same time is also good news too),
well, I integrated my MarkupColorFold into Lazarus  :-X :D :-*


The good news is there are no error, it was gracefully integrated (debug assert enabled).
The bad news is: it is not as what it was designed. Vertical lines are jerk here.
Rather helps programmer, it will confusing them.  %) 
Yeah, It need some finishing/touch-up to be looked as cool enough...
(see attachment)


This messy visual effect, can be solved by drawing all vertical lines,  straight to "end;" keyword vertically.
We can see how CnPack solves the same problem. (see attachment II )


-----
Using MarkupWordGroup as guidance, we can easily found the 'end' part of each fold block.
When the 'end' is founded, we can compare with the 'begin' positions, resulting which one is the left most.
this leftmost position will be used to draw the correct vertical lines.
This way, one vertical-line-mark is done.
But if a line is located in 3th dept of folds, this step should repeated 3 times.


But, because MarkupColorFold will operate per sourcecode's line basis,
the next sourcecode-line will repeat comparing the left most pair of that begin~end (that have already proir calculated)
... plus : if it still in 3th depth of folds, comparison will be repeated 3 times again.


see?


For honest, the longest procedure AFAIK would be only 200 lines including off-of-screen lines.  (sorry for 10K abuse).
But, couldn't we make this real solution more efficient?


MarkupWordGroup only finding pairs of begin~end once, or twice, or maximum 3 time, and her job done.
But MarkupColorFold is more expensive on looking for paired begin~end than MarkupWordGroup in this situation.


So, it is my assumption : storing both closing+opening tag of a fold-block will minimize allocate/deallocate memory,
which in turn will reduce your battery-laptop consumption.


How?


THere is a good idea that depends on the concept/fact that HL store states for each line.
Assumed that System.pas has 10K lines, it should have 10K states stored, right? because 1line = 1 states.
Hence, on file loading, when a 'begin' of procedure block founded, I can store that begin postion.
when 'end' of that procedure founded, I stored that in the shared-folding-block (whatever Range / CodeFoldBLock?).
Now, when SynEdit need to refresh the screen, MarkupFoldColorPascal wouldn't need to find the left most X of paired 'begin'~'end' for each dept of fold block.
She can directly read the data stored for each block, and when need to read the same info of .Parent block just read, without re-searching.

(similar situation was described here: http://forum.lazarus.freepascal.org/index.php/topic,30122.msg194828.html#msg194828 (http://forum.lazarus.freepascal.org/index.php/topic,30122.msg194828.html#msg194828)
| |    | | | HorzEntry := EMPTY_ENTRY;
(see attachment for complete lines)


see?

You might say that we don't really want to store this specific pascal things in MarkupFold,
Well, For reason of general purpose use, let MarkupColorFold as is.
For pascal, we can inherit it into new class that able to store 3 x integer x 2(begin,end) = 6 integer length per block (not per line).


If this concept is negative, how can an HL rescan (reparsing) a line randomly?




But it most probably impossible, when range (or foldblock?) is reused rewriten unpredicably.  :(

Ranges (if they are an object) are re-used. And with that the same FoldBlock object is used to represent the state on many lines.
Code: Pascal  [Select][+][-]
  1. begin                   // range at the end of line has foldblock 0x0001
  2.   if a then begin    // range at the end of line has foldblock 0x0002
  3.   end;                  // range at the end of line has foldblock 0x0001 since the inner fold is closed
  4.   if longer_condition then begin // range at the end of line has foldblock 0x0002 again (re-used), even the X pos differs.
  5.   end;
  6. end;
  7.  
What do you think ? 8)
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on December 11, 2015, 08:26:52 pm
Still reading up....

But why: LogVertGuideX ? Shouldn't that be calculated in your markup?
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on December 11, 2015, 08:39:16 pm
Also, do you still change the "exit(-1)" to 0 in the nodelist (if not valid).

Some code currently relies on -1 (the example foldhl will not fold, if it is 0)

I agree the -1 is bad. So it is also possible to add a property IsValid and change all code that depends on it.

So one of 2 choices:
1) Make your code work with -1
2) add IsValid. This must be a standalone patch (or one separate commit of all involved changes in your git). Then it can be merged as a standalone svn commit.
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on December 11, 2015, 08:50:59 pm
about the files you changed from original.

Does your git has the unmodified versions too, so on could simply compare against it?
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on December 11, 2015, 10:14:24 pm
Some of the older changes:

Code: Pascal  [Select][+][-]
  1.     FIsCollectingNodeInfo: boolean;
  2.     FFoldNodeInfoList: TLazSynFoldNodeInfoList;
  3.     FCollectingNodeInfoList: TLazSynFoldNodeInfoList; //x2nie not sure if it is needed as here will be 2 TLazSynFoldNodeInfoList
  4.  

There should only be one list.

FIsCollectingNodeInfo should be available through read only property, so pas HL does not need its own.
It then must be set, and cleared (try finally) in the base class.

Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on December 12, 2015, 12:11:15 am
Quote
Code: Pascal  [Select][+][-]
  1. LogX2 := LogX1 + L -1;
But...
the MarkupWordGroup seem as wrong on Node.LogXEnd, mean conflict with my idea.
See the "food"~"/food"  vs  "repeat"~"until" screengrab.

IIRC (need to check) end is the begin of next token.
Think of the pos as pointing between two chars, not the start of a char.

Code: Pascal  [Select][+][-]
  1. foo bar
foo starts at 1
last char of foo is (starts at) 3
foo ends at 4
space starts at 4

IIRC that is common on many parts of SynEdit.

Keep MWG, by above logic.
Btw, this is also where the caret would be, if it was at the end of foo.


Quote
please ignore my previous question
too late



Quote
Now, the problem is when text/line trailing with tab. The vertical lines drawn improperly.
Either:
1)  they are past end of line. But selection (in column mode) can do it, so there must be a way.

2) logical vs physical http://wiki.lazarus.freepascal.org/SynEdit#Logical.2FPhysical_caret_position
if there are #9, or multybyte utf8 (umlauts äüö, accents, Chinese, ....)

You need to store the phys pos, as that is the same in all lines. logical changes.

Need to check later




Hey, I currently face a real problem ... >:D  !
Quote
This messy visual effect, can be solved by drawing all vertical lines,  straight to "end;" keyword vertically.
We can see how CnPack solves the same problem. (see attachment II )

relates how to the quote of mine that you posted before?

see my comment above.

It should work with the frame for now. There may be shlight changes, but they would still need to go line by line

cnpack probably paints on top of all (and I have seen pictures where the yellow "begin" was painted with an offset to the original....)

But in Lazarus SynEdit composes colors by priority. A user can define another markup (eg selection) to have higher priority. All other markups are evaluated per line, so yours must be too.




Quote
... plus : if it still in 3th depth of folds, comparison will be repeated 3 times again.
That is why this code (searching end line) should be added to nested fold list. there known info can be cached and reused.

Quote
For honest, the longest procedure AFAIK would be only 200 lines including off-of-screen lines.  (sorry for 10K abuse).
But, couldn't we make this real solution more efficient?
Yes, but not by storing on the range.
see above: cache in the list.

I have some ideas there, but I need more time to evaluate and explain.

In the nested list, if you change the line, it can still keep the previous data as cache, and only calculate until it can use the cache.

This cache would be per module, It will work well just for your markup, and it will work well for other markup, but it will not add synergy between markups. For that another cache may be needed, but that can be done later.

Quote
But, because MarkupColorFold will operate per sourcecode's line basis,
the next sourcecode-line will repeat comparing the left most pair of that begin~end (that have already proir calculated)
... plus : if it still in 3th depth of folds, comparison will be repeated 3 times again.
Not if cached in the nested list. Also I have to look at your code again. I need to see if the invalidate issue was truly fixed.
And depending on this other changes may make this partly obsolete.

Quote
Assumed that System.pas has 10K lines, it should have 10K states stored, right? because 1line = 1 states.
As I wrote a few posts back: No.

Example pas HL, it will probably have 1500 range objects for the 10k lines. (and the ratio gets better, if you have 100k lines).
For each line it has a pointer entry, that points to one of those objects.

And that is the reason they must be immutable, because several lines use them, if you changed it, that has side effects on other lines.

Quote
Hence, on file loading, when a 'begin' of procedure block founded, I can store that begin postion.
when 'end' of that procedure founded, I stored that in the shared-folding-block (whatever Range / CodeFoldBLock?).
See above, you need a lot more range objects (and folding blocks, the 2 go together)

You would also need to fix, that very long ago (maybe even before synedit was added to Lazarus) someone decided that range objects are only freed when the HL is freed. They are kept, even if they no longer apply to any line.
Currently this is at best a very minor issue, but the more of them are created, the bigger it gets.

You still may need to find the end line. it could contain an "invalid flag" or other info. Unless you store a copy of the entire node info record on the fold block. But that would be a lot of data, AND also mean you have to calculate all this data during every scan, even for lines that are currently not visible.
Not tested, but try (forward declaration):
Code: Pascal  [Select][+][-]
  1.   type TFoo = class
  2.   // 10 lines
  3.   ;
  4.  
An maybe also "class of".

If the search is improved correctly, then you do only one scan independent on nesting depth, and the data is re-used for the next (and further) line(s).

And all the search will be abstracted in the nested list. So any optimization can be done now or later, it does no longer affect your code. Your code asks the list. the list can get it whatever way, and the way can even be changed.

Quote
If this concept is negative, how can an HL rescan (reparsing) a line randomly?
For rescanning the x pos is not needed. The xpos is only needed outside the HL.

Quote
But it most probably impossible, when range (or foldblock?) is reused rewriten unpredicably.
it is re-used (not unpredictable though).



Overall, I do see what you mean. I don't currently think it is the way to go.
In any case if the problem can be abstracted into a few re-usable helper functions, then it can be changed later if indeed needed.

--------
EDIT
Also adding more data to the HL, one needs to look at who/what benefits.
Some users use the HL only for highlight, no fold, no markup. Then the data is wasted.
Of course then for some HL the fold level is wasted too. But that would be all the more reason not to add to it (pascal needs parts of the fold info, because some symbols change. For examle "of" looks at the foldtype, "case of" means a label is next, but only in  a case block. Also "Cdecl", "final" and others need them.


-----
EDIT2:

I am NOT asking you to write the cache code etc. (you are welcome, but it would be a separate project)

Just add to the TLazSynEditNestedFoldsList:
Code: Pascal  [Select][+][-]
  1.     property EndNodeLine[Index: Integer]: Integer read GetEndNodeLine;        
  2.  
that does the loop on the foldlevel.

So that EndNodeLine[n] returns the line that matches the open-node in NodeLine[n]

Ideally, if EndNodeLine[n-x]  is known, continue from there.

then maybe also add
Code: Pascal  [Select][+][-]
  1. property HLEndNode[

same as HLNode just the closing.
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on December 12, 2015, 12:25:14 am
To start moving some code to svn.
Code: Pascal  [Select][+][-]
  1. Also, do you still change the "exit(-1)" to 0 in the nodelist (if not valid).
  2.  
  3. Some code currently relies on -1 (the example foldhl will not fold, if it is 0)
  4.  
  5. I agree the -1 is bad. So it is also possible to add a property IsValid and change all code that depends on it.
  6.  
  7. So one of 2 choices:
  8. 1) Make your code work with -1
  9. 2) add IsValid. This must be a standalone patch (or one separate commit of all involved changes in your git). Then it can be merged as a standalone svn commit.


And LogVertGuideX ?

Then I can extract the code, and merge it.




Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on December 12, 2015, 12:28:03 am
LogVertGuideX

1) I wouldnt make it part of the getter.
If a HL needs it, then it must override initializeNode (or what the name is).

2) I can see the xml hl needs more than the token.

But does it not also need an end?
Code: XML  [Select][+][-]
  1. <foo val=1>
what would be highlighted?

Also the name needs to be more generic. Other markup may use it too.

Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on December 12, 2015, 06:47:43 pm
Quote
Hence, on file loading, when a 'begin' of procedure block founded, I can store that begin postion.
when 'end' of that procedure founded, I stored that in the shared-folding-block (whatever Range / CodeFoldBLock?).

In addition to my previous reply. This info would not be useful, not even to your code.

The HL does by design only know logical positions. Changing this would be changing almost the entire design of SynEdit.

But for painting you need the physical pos. To get that you need the actual lines (line with begin and line with end). And to find the lines you still need to do the search on foldlevel.

And no: you cant store the linenumbers of those lines. If you do you must update all ranges if a line is inserted or deleted (at top of file). Currently you only scan the inserted line (in most cases)

EDIT:
In fact it is not possible to have phys pos in HL. if 2 synedit are linked, they share the HL, and the share the same ranges. But each SynEdit can have different tab settings, meaning different phys pos for the same log pos
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: x2nie on December 12, 2015, 07:57:19 pm
Okay, here is my today update:
* IsCollectingNodeInfo has been set as readonly property (currently no affect to pas HL)
* FNodeInfoList hasbeen deleted, in performing of new name (similar purpose) of FCollectingNodeInfoList,
   I think the new one is more intuitive.
* LogVertGuideX has been deleted. Its not really needed, so far.
   As For XML, the difference between tag location and vertical guide, possibly can be adjusted in XML HL,, later.





Also, do you still change the "exit(-1)" to 0 in the nodelist (if not valid).

Some code currently relies on -1 (the example foldhl will not fold, if it is 0)

I agree the -1 is bad. So it is also possible to add a property IsValid and change all code that depends on it.

So one of 2 choices:
1) Make your code work with -1
2) add IsValid. This must be a standalone patch (or one separate commit of all involved changes in your git). Then it can be merged as a standalone svn commit.
This is weekend, I am at home with my LiteZarus (compatible = Lazarus 1.5).
still using with  "-1", the demo is fine without error,
but when integrating with IDE, there are error every time.
I am not sure about it now, probably you can test in your Lazarus 1.7 ?


As for merging my contribution (SynEditHighlighterFoldBase) into lazarus svn ,
1) so far so good. One I worried about is FConfigFold things. I added a guard (check if nil else) but I never test it yet.
2) sorry, I can provide files *.pas, no patches. If you really want to see diff, just replace / copy-paste them into your lazarus codes.

Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on December 12, 2015, 08:04:18 pm
Quote
2) sorry, I can provide files *.pas, no patches. If you really want to see diff, just replace / copy-paste them into your lazarus codes.

I can work from your git.

But if you add modification to more files, please commit the unmodified copy first, then I can diff against that.
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: x2nie on December 12, 2015, 08:41:05 pm
Quote
2) sorry, I can provide files *.pas, no patches. If you really want to see diff, just replace / copy-paste them into your lazarus codes.

I can work from your git.

But if you add modification to more files, please commit the unmodified copy first, then I can diff against that.
Oh? :o  okay, I will. I just didn't think that it was important for people (other than me).
Also, it was a bit difficult for me (copying unmodified files first). but now I have another idea to automate that. So don't worry, if I did that accidently, git support the "undo commit"  8)  I can resolve them.


Do you still need unmodified files available before any changes made to them? if so, I can "git rebase" to re-flow my commit hystories.  :-\ :D ;D
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on December 12, 2015, 09:04:33 pm
I use a local merge tool now.
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on December 12, 2015, 10:59:53 pm
while merging I noted a few things:

1) my fault, we need 2 variables for 2 lists, I put them back already

2) TSynCustomFoldHighlighter.CollectNodeInfo
Removing comments,
Removing sfaOpenFold, sfaCloseFold => deprecated
Code: Pascal  [Select][+][-]
  1.     if not LevelChanged then
still needed? this should be covered by blockenabled. And not sure if a good idea if only on closing.

3) I need to review IncreaseFoldLevel, and the 2nd depth counter.
More on that later

4) sfaMarkup needs to be added to each foldconfig in GetFoldConfigInstance of each HL.


Code: Pascal  [Select][+][-]
  1. procedure TSynCustomFoldHighlighter.CollectNodeInfo(FinishingABlock: Boolean;
  2.   ABlockType: Pointer; LevelChanged: Boolean);
  3. var
  4.   //DecreaseLevel,
  5.   BlockTypeEnabled: Boolean;
  6.   act: TSynFoldActions;
  7.   BlockType: Integer;
  8.   nd: TSynFoldNodeInfo;
  9. begin
  10.   if not IsCollectingNodeInfo then exit;
  11.  
  12.   BlockTypeEnabled := False;
  13.   if (ABlockType <> nil) and (PtrUInt(ABlockType) < FoldConfigCount) then
  14.     BlockTypeEnabled := FFoldConfig[PtrUInt(ABlockType)].Enabled;
  15.  
  16.   //Start
  17.   if not FinishingABlock then
  18.   begin
  19.     act := [sfaOpen, sfaOpenFold]; // todo deprecate sfaOpenFold
  20.     if BlockTypeEnabled then
  21.       act := act + FFoldConfig[PtrUInt(ABlockType)].FoldActions
  22.     else
  23.       act := act + [sfaFold];
  24.     DoInitNode(nd, FinishingABlock, ABlockType, act, True);
  25.   end
  26.   else
  27.   //Finish
  28.   begin
  29.     act := [sfaClose, sfaCloseFold]; // todo deprecate sfaCloseFold
  30.     if BlockTypeEnabled then
  31.       act := act + FFoldConfig[PtrUInt(BlockType)].FoldActions
  32.     else
  33.       act := act + [sfaFold];
  34.     if not LevelChanged then
  35.       act := act - [sfaFold, sfaFoldFold, sfaFoldHide];
  36.     act := act - [sfaFoldFold, sfaFoldHide]; // it is closing tag
  37.     DoInitNode(nd, FinishingABlock, ABlockType, act, LevelChanged);
  38.   end;
  39.  
  40.   FCollectingNodeInfoList.Add(nd);
  41. end;
  42.  
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on December 12, 2015, 11:03:36 pm
btw, in your sample:
- indent a "begin" (vert line moves)
- cursor down 10 lines
- undo
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on December 12, 2015, 11:26:57 pm
Now here is the issue (and I didnt realize it before)

When you copies PasFoldLevel that was actually right (-ish), but not because the value is needed by the markup.


Now in pas hl we have 2 counters
1) NESTED for all nodes, even if you cant fold the node. (e.g. cfbtIfThen)
2) FOLD only those you can fold

This gives the fold module a slight advance on the search.
If a line contains an "if" and therefore a none foldable node, the 2nd of the 2 levers does not change. The search skips over that line.

The markup needs to seach such lines too, because a markup line may not be foldable.

Now one could argue: Add a MarkupLevel too.
Then that is 3 levels (if all markups share a level, even if they do not share all nodes, otherwise it is even more)

Probably in future other subsets of nodes want there level too. So then 4 or 5 levels.
Each new combination of levels creates a new range. ouch. Not a good way to go.

So for now (until the parsing code relying on it can be fixed: keep the 2 levels.
Actually the (1) "NESTED all" does not matter. It equals the depth of the tree via the parent. So it never creates a range that does not exist already anyway.

EDIT: except the associated "min level" creates new nodes....


Later:
On the long run the "FOLD" can be removed. The search then has to look at the TSynCustomCodeFoldBlock.BlockType. (and the foldconfig). This still works without scanning the line. (even improves current markup search)

If the blocktype is configured for the correct action then the line is a match.
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on December 13, 2015, 07:36:15 pm
some mem leaks / there are more

Code: Diff  [Select][+][-]
  1. diff --git a/syneditmarkupfoldcolors.pas b/syneditmarkupfoldcolors.pas
  2. index bec8f81..72b4e1d 100644
  3. --- a/syneditmarkupfoldcolors.pas
  4. +++ b/syneditmarkupfoldcolors.pas
  5. @@ -338,6 +338,7 @@ begin
  6.        inc(i);
  7.        //dec(i);
  8.    end;
  9. +  Nest.Free;
  10.  end;
  11.  
  12.  procedure TSynEditMarkupFoldColors.PrepareMarkupForRow(aRow: Integer);
  13. diff --git a/syngutterfolddebug.pas b/syngutterfolddebug.pas
  14. index 8844ded..d9f622e 100644
  15. --- a/syngutterfolddebug.pas
  16. +++ b/syngutterfolddebug.pas
  17. @@ -153,6 +153,7 @@ var
  18.  
  19.    finally
  20.      TextDrawer.EndDrawing;
  21. +    nest.Free;
  22.    end;
  23.  
  24.  end;
  25.  
  26.  
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on December 13, 2015, 09:36:48 pm
ok, so I merged TSynCustomFoldHighlighter to svn.

Quote
Revision: 50779
Message:
SynEdit: add FoldNodeInfo to TSynCustomFoldHighlighter / prepare new markups. Patch by x2nie
Author: martin
Date: 13 December 2015 20:36:29
----
Modified : /trunk/components/synedit/synedithighlighterfoldbase.pas

I restored having 2 fields (after I incorrectly said only one was needed), I restored the 2nd foldlevel (as it will be needed for now)

The " if not LevelChanged then" is not needed. Checking the code in pas hl, it should be the same as foldblock.enabled and foldblock.actions.


This will also mean the following can thanks to you get resolved
http://bugs.freepascal.org/view.php?id=23016
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: x2nie on December 14, 2015, 05:52:22 am
well, before we continue further, I think let's review your modification.
You have broken the basic rule.


- The foldHL can't be collapsed/folded.
- My ColorFold-HighLighter  too.
- The MarkupWordGroup doesn't work any longer.
Do you miss something?

But, any of other HL (in my demo) works:
+ Pas HL work
+ LFM highligter work
+ XML HL work


I guess there are around ConfigFold (that I worried yesterday)


I am using Lazarus 1.7
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: x2nie on December 14, 2015, 06:24:43 am
Oh, it was easier than I thought.
It just need to add 'sfaFoldFold' and collapsible will be back again there.



Code: Pascal  [Select][+][-]
  1.   //Start
  2.   if not FinishingABlock then
  3.   begin
  4.     act := [sfaOpen, sfaOpenFold]; // todo deprecate sfaOpenFold
  5.     if BlockTypeEnabled then
  6.       act := act + FFoldConfig[PtrUInt(ABlockType)].FoldActions
  7.     else
  8.     if not BlockConfExists then
  9.       act := act + [sfaFold,sfaFoldFold, sfaMarkup]; //<<---------------here
  10.     DoInitNode(nd, FinishingABlock, ABlockType, act, True);
  11.   end
  12.   else
  13.   //Finish
  14.   begin
  15.     act := [sfaClose, sfaCloseFold]; // todo deprecate sfaCloseFold
  16.     if BlockTypeEnabled then
  17.       act := act + FFoldConfig[PtrUInt(ABlockType)].FoldActions
  18.     else
  19.     if not BlockConfExists then
  20.       act := act + [sfaFold, sfaMarkup]; //<<---------------------- and here. +markup?
  21.     act := act - [sfaFoldFold, sfaFoldHide]; // it is closing tag
  22.     DoInitNode(nd, FinishingABlock, ABlockType, act, LevelChanged);
  23.   end;                    
  24.  
                                     


Anyway, I agree to get rid the 'sfaMarkup' everywhere. It should be calculated automatically, right?
sfaMarkup seem as not that special case. Every opening~closing fold tag can be assumed as markupable by MarkupWordGroup (and other need if any).
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on December 14, 2015, 08:12:52 am
ups sorry, I missed that.

I am going to add sfafoldfold. (just need to test again, so in a couple of hours).

Basically "folding" (or sfaFold, sfaFoldFold) is the default for HL that do not specify what a they want.

sfaMarkup can only work if there is a foldconfig.

Sample from xml hl
Code: Pascal  [Select][+][-]
  1. function TSynXMLSyn.GetFoldConfigInstance(Index: Integer): TSynCustomFoldConfig;
  2. begin
  3.   Result := inherited GetFoldConfigInstance(Index);
  4.   Result.Enabled := True;
  5.   if TXmlCodeFoldBlockType(Index) in [cfbtXmlNode] then begin
  6.     Result.SupportedModes := Result.SupportedModes + [fmMarkup];
  7.     Result.Modes := Result.Modes + [fmMarkup];
  8.   end;
  9. end;
  10.  
  11. function TSynXMLSyn.GetFoldConfigCount: Integer;
  12. begin
  13.   Result := 1; // ord(high(TXmlCodeFoldBlockType)) - ord(low(TXmlCodeFoldBlockType));
  14. end;
  15.  

You can set the count to 1.

Then you can still pass nil (which translates to zero; and zero is the only element in a list with a count of 1)

or you define an enum
Code: Pascal  [Select][+][-]
  1.  TMyHlCodeFoldBlockType = (
  2.     cfbtFooBar
  3. // , optional more
  4.   );
  5.  
and call
Code: Pascal  [Select][+][-]
  1. StartCodeFoldBlock(Pointer(ptrUint(ord(cfbtFooBar))), true)
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on December 14, 2015, 08:35:45 am
foldconf:

for each node type (or actually for the amount returned by GetFoldConfigInternalCount) a config object will be created.

SupportedModes
The HL decides if a node can have a mode or not. E.g. the pas hl only allows "hide" for comments. (In this case its not a technical reason, just seemed sensible)

Modes
initialized with the default. This can be changed by user.

Enabled
must be true, or the node is entirely ignored (or at least should)


currently a node can have the following modes (set of)

fmFold: normal fold
fmHide: fold including hiding the line itself (comments in pas, can be configured)
fmMarkup: the word at caret matching node markup

and I suggest, that you add
fmMarkupOutline 
then the 2 markup modules (yours and the existing) can have different conf.

and use this in your markup, instead of sfMarkup

of course you dont use fmMarkup, but sfaMarkup
So you also need to create an new action sfaMarkupOutline.

Look at TSynCustomFoldConfig.SetModes  that should tell you how.

to make  them easier to remember:
fm = fold mode
sfa = syn fold action
cfbt = code fold block type


TODO in the code

GetFoldConfigInternalCount in the base class should probably default to GetFoldConfigCount, so a child class need only that one (but can have both)

SupportedModes should not be public. It could be set when calling the constructor. Or the TSynCustomFoldHighlighter (same unit, can access protected) could have itself a protected method to forward the data.
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: x2nie on December 14, 2015, 12:12:39 pm
OK, 2 bits of info.

1) foldconfig.

currently length is set in .Create. not good, maybe move.

GetFoldConfigInstance
gets the initial setting.

for each foldtype (cfbtBeginEnd,...) it indicates, if it can
Code: Pascal  [Select][+][-]
  1. TSynCustomFoldConfigMode = (fmFold, fmHide, fmMarkup);
- fold
- hide
- markup (word triplet at caret)

you could add fmMarkupOutline then this can be configured, and the HL can indicate which nodes are for your module.

the HL can set 2 flags (maybe more)
- sfaMarkupOutline
  visible outline
- sfaMarkupOutlineIndend
  not outlined, but increase color (e.g. a nested procedure block)

those would then be set in HL.InitNode() according to the language rules.

As for the "level" of intend. The HL levels are not correct, but when you go through Nest : TLazSynEditNestedFoldsList; you could keep count of nodes that have the flag (you need to start left to right then).

Or simply set them on the result FHighlights. But then you need to add dummy entries for the invisible sfaMarkupOutlineIndend. So that is not desirable.
....
Okay, I learn by doing. I am trying  to understand what you said.
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: x2nie on December 15, 2015, 02:38:44 pm
Okay, I did it! ;D

Further, after I know how to work with Node,
 I also found a bugfix of wrong triplet "if-then",  when there is no "else" and there is nor ';'
The bug is: the triplet will be paired with it's parent's 'end', which it should only 'if-then' highlighted.
Code: Pascal  [Select][+][-]
  1. function TSynPasSyn.Func23: TtkTokenKind;
  2. var
  3.   tfb: TPascalCodeFoldBlockType;
  4.   sl : integer;
  5. begin
  6.   if KeyComp('End') then begin
  7.     if ((fToIdent<2) or (fLine[fToIdent-1]<>'@'))
  8.     then begin
  9.       Result := tkKey;
  10.       fRange := fRange - [rsAsm, rsAfterClassMembers];
  11.       PasCodeFoldRange.BracketNestLevel := 0; // Reset in case of partial code
  12.       // there may be more than on block ending here
  13.       tfb := TopPascalCodeFoldBlockType;
  14.       while (tfb in [cfbtIfThen]) do begin // no semicolon before end
  15.         sl := fStringLen;
  16.         fStringLen:=0; // <--------------------------------- here bugfix
  17.         EndPascalCodeFoldBlock(True);
  18.         fStringLen := sl; // <------------------------------- here bugfix
  19.         tfb := TopPascalCodeFoldBlockType;
  20.       end;                  
see attachments



Now, my problem is: How to make if-then to not increase level?
I want the if-then---begin-end---else---begin-end all of them in similar color. I think it should be in similar level, isn't it?

edit:
see secondary attachment, if~then and it's begin~end has different color. How can I make them in one single color?
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on December 15, 2015, 11:35:07 pm
In pas HL
Code: Pascal  [Select][+][-]
  1.    StartPascalCodeFoldBlock(cfbtIfThen, not( TopPascalCodeFoldBlockType in [cfbtCase, cfbtIfThen]));
  2.  

not tested, but that probably breaks case-else
Code: Pascal  [Select][+][-]
  1. case a of
  2.   1: foo;
  3.   2: if b then
  4.         if c then
  5.           foo1
  6.         else // NOT highlighted as case-label
  7.           foo2
  8.       else  // NOT highlighted as case-label
  9.         foo3
  10.   else  // highlighted as case-label
  11.     bar;
  12. end
  13.  

You probably want to look at initnode, and remove the sfaOutline for nested cases.
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: x2nie on December 16, 2015, 03:28:26 am
@Martin, in english, please?


Are you pointing to bugfix I show above,
or are you answering my question,
or you show me another bug?



About the code in your last post:

Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on December 16, 2015, 04:43:40 am
Quote
Are you pointing to bugfix I show above,
or are you answering my question,
or you show me another bug?

I saw that line of code when I diffed your latest commits.

I did not test, but suspect that it might interfere with existing features.



I was not talking of your color only

the pas dl has an option to highlight all case labels (in the IDE you can configure this / I use a frame around them).

Having nested "if" blocks means keeping track of where an else belongs.

I have not tested your change. But it may break this existing behaviour.



As for your colors. You do not highlight the case labels 1: or 2: so the "else" of the case should also be ignored.




There is a bigger problem:
Code: Pascal  [Select][+][-]
  1. ANode.FoldLvlEnd

1) ANode.FoldLvlEnd may be removed in future

2) never mind the future. In the IDE you can configure what folds, and what does not fold. You can disable folding for "begin" blocks.
If you do, then for a begin block the level will no longer increase. So if you have nested begin blocks (nested loops) then all begin will have the same color.

I said it before: the level is not intended for what you do. It only server finding opening and closing lines.

When you prepare FHighlights, then you know how many outer lines there are. So you can set the colors from the content of FHighlights.

However you have special cases.
1) "If" No increase in color wanted
add an action (to the inner) sfaOutlineMergeParent.

2) extra increase ("begin" in nested proc)
add an action sfaOutlineExtraLevel

those are just actions. there are no new modes.

The HL can add them in InitNode. (or somewhere like that)


Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on December 16, 2015, 05:40:49 am
Quote
1) "If" No increase in color wanted
add an action (to the inner) sfaOutlineMergeParent.

2) extra increase ("begin" in nested proc)
add an action sfaOutlineExtraLevel

You may not even need 2.

If you make procedure and begin merge colors, so you have one long line from procedure, over begin to end, in one color, then nested procedures already have their own color.

Though if you prefer to start the line at begin, then add it.

It can later become an options

Code: Pascal  [Select][+][-]
  1. SynPasSyn.OutlineOptions = (
  2.    mergeProcBegin, // if both proc and begin have fmOutline
  3.    mergeIfBegin, // if both IF and begin have fmOutline
  4.    mergeNestedIf // only one color for all if
  5. )
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: x2nie on December 16, 2015, 08:20:15 am
If you make procedure and begin merge colors, so you have one long line from procedure, over begin to end, in one color, ...
Excellent. That is what I really wanted.


I also would reapply that rule into the if-then make if the longest line, over the then,begin and else,begin. whooaa..... :-[
The reason is: I don't really want to fold/unfold at "then" (current rule), instead I want at "if". I believe other people want too.
* Was it a mistake to fold/unfold at "then". ? :P


If it was reached, I can work for another keywords: do~while, with~do, for~do ...etc.
but now the hardest part is to change the behavior of if~then~else. So, other keywords will follow.




About the pascal HL, I use both original and experimental (in case you aren't noticed, ) to compare side-by-side.
My experimental PAS Highlighter, is to reuse our modified base class. (but I seem failed to merge InitNode + DoInitNode).
The file is SynHighlighterMiniPas2.pas

So, if you worry about whether it will interferer with existing feature, you might right,
it may completely break many rules. But, this way is easier way I can do during learning.
We can discuss my modification PAS HL to back obey the IDE rules, maybe later.

What I should do with SynPasSyn.pas? I think I will replace it with mine when the experiment'r result is (someday) okay.

Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: x2nie on December 16, 2015, 08:31:30 am
I said it before: the level is not intended for what you do. It only server finding opening and closing lines.
Okay, deal. It's clear that I was in wrong direction when depends on fold-level for nest-coloring, it should depends on other counter.
I may need to add that counter into PasRange.?

Quote
When you prepare FHighlights, then you know how many outer lines there are. So you can set the colors from the content of FHighlights.
:o You have said it several time, but sorry for my stupid, I don't understand yet either.  Maybe, I should just go forward, forgeting it for now.





I am still trying
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: x2nie on December 16, 2015, 12:02:13 pm
As for your colors. You do not highlight the case labels 1: or 2: so the "else" of the case should also be ignored.


Okay, it was a bug for my coloring markup over PAS hl bug too. So, if another SynEdit module working with it, they will found the same bug.
I've fixed.


Code: Pascal  [Select][+][-]
  1. function TSynPasSyn.Func41: TtkTokenKind;
  2. begin
  3.   if KeyComp('Else') then begin
  4.     Result := tkKey;
  5.     if (TopPascalCodeFoldBlockType = cfbtIfThen) then
  6.       EndPascalCodeFoldBlock
  7.     else
  8.     if TopPascalCodeFoldBlockType = cfbtCase then begin
  9.       StartPascalCodeFoldBlock(cfbtCaseElse);
  10.       FTokenIsCaseLabel := True; //<-------------------------------bug
  11.     end;



Code: Pascal  [Select][+][-]
  1.     if TopPascalCodeFoldBlockType = cfbtCase then begin
  2.       FTokenIsCaseLabel := True; // <--------- it should be there before calling StartBlock
  3.       StartPascalCodeFoldBlock(cfbtCaseElse, True);
  4.  



edit:
Fully configurable.
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: x2nie on December 16, 2015, 01:34:11 pm
However you have special cases.
1) "If" No increase in color wanted
add an action (to the inner) sfaOutlineMergeParent.

OH! I know what you mean. I've modified original PAS HL a little bit.
The original is far worst (if & then is in different color).
But IMHO, My PAS (SynMiniPas2.pas) is also relevant to has sfaOutlineMergeParent. give me a try..

please compare this (original pas) screengrab with previous (my mod pas) one.
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on December 16, 2015, 04:55:03 pm
Quote
When you prepare FHighlights, then you know how many outer lines there are. So you can set the colors from the content of FHighlights.
:o You have said it several time, but sorry for my stupid, I don't understand yet either.  Maybe, I should just go forward, forgeting it for now.

Lets start with this one.

If I read your code correctly:

Step 1:
you add all nodes to FHighlights.
First from the current line, then from TLazSynEditNestedFoldsList. As a result the innermost node is at index 0.
You will have to add duplicates at the same x pos, so you can NOT
    if FHighlights.X = x then    exit;
but instead add a flag to this node.

Then before you sort them (SortLeftMostFI), do something like
Code: Pascal  [Select][+][-]
  1. a := 0
  2. for i = high(FHighlights) downto 0 do begin// start at the outermost)
  3.   FHighlights[i].color = a mod lenght(colors).
  4.   inc(a)
  5. end
  6.  

And you got your colors. Of course if you want to join 2 lines, do not inc the color.

And that way you do not need any level from the HL at all.

Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on December 16, 2015, 04:58:34 pm
Last time I checked (a few days ago) you still didn't have working invalidation.

1) add spaces at a line that contains a begin, the vert line moves.
2) place caret 10 lines below
3) hit undo.

But similar with codetools, and linked editors.

You may need to keep a list for all visible lines where you painted, and then on DoTextChanged find the lines that changed and invalidate

Code: Pascal  [Select][+][-]
  1.   if EndLine < 0 then exit; //already refreshed by syn
  2.  
Dont worry. Invalidating already invalidated lines is no problem. Synedit catches it in some cases, otherwise the OS handles it as null op.
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on December 16, 2015, 05:13:57 pm
Quote
I also would reapply that rule into the if-then make if the longest line, over the then,begin and else,begin. whooaa..... :-[
The reason is: I don't really want to fold/unfold at "then" (current rule), instead I want at "if". I believe other people want too.
* Was it a mistake to fold/unfold at "then". ?

yes and no. fold rules may need to be changed. But the current one are there for a reason.

If (and before) they get changed, the testcase (folder test) must contain sufficient tests, that they work in all variations and configs.

To merge the begin/end in the else, a foldblock for the else must be added (terminated by either ";", end, until, ....)

Similar foldblocks for while and others may be needed (and only enabled, if your markup is active / that is if they have sfaOutline, because then it is assumed that a markup is active). Maybe a global switch.... but not important now.

For your test HL that is ok. then again since all is configurable your code needs to do something if they are absent too.



merging doInitNode to pas hl is fine, but it is a completely independent task. so it should not mix with any of the other changes. If there is a stable diff for that then it can be merged.



Quote
Code: Pascal  [Select][+][-]
  1.       StartPascalCodeFoldBlock(cfbtCaseElse);
  2.       FTokenIsCaseLabel := True; //<-------------------------------bug
  3.  

Why is it a bug, where does it go wrong?



I look at the latest changes later

Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on December 16, 2015, 06:06:33 pm
Code: Pascal  [Select][+][-]
  1.     if not BlockConfExists then
  2.       act := act + [sfaFold,sfaFoldFold, sfaMarkup, sfaOutline];
  3.  
You will eventually have to add foldconf to any HL that should support this.
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: x2nie on December 17, 2015, 03:45:07 am
Code: Pascal  [Select][+][-]
  1.       StartPascalCodeFoldBlock(cfbtCaseElse);
  2.       FTokenIsCaseLabel := True; //<-------------------------------bug
  3.  

Why is it a bug, where does it go wrong?


The order.
FTokenIsCaseLabel is seemly needed to distinct whether an identifier is a Case's Label or not.
If it NOT set to True, then in DoInitNode the identifier is not related to foldConfig. which is a bug. DoInitNode is called in every StartPascalCodeFoldBlock( ).

The bug is probably not visible, but when using my markup, there is obviously a bug.
.see my complete post, for correction of the order.
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: x2nie on December 17, 2015, 03:52:45 am
Quote
When you prepare FHighlights, then you know how many outer lines there are. So you can set the colors from the content of FHighlights.
:o You have said it several time, but sorry for my stupid, I don't understand yet either.  Maybe, I should just go forward, forgeting it for now.

Lets start with this one.

If I read your code correctly:

Step 1:
you add all nodes to FHighlights.
First from the current line, then from TLazSynEditNestedFoldsList. As a result the innermost node is at index 0.
You will have to add duplicates at the same x pos, so you can NOT
    if FHighlights.X = x then    exit;
but instead add a flag to this node.

Then before you sort them (SortLeftMostFI), do something like
Code: Pascal  [Select][+][-]
  1. a := 0
  2. for i = high(FHighlights) downto 0 do begin// start at the outermost)
  3.   FHighlights[i].color = a mod lenght(colors).
  4.   inc(a)
  5. end
  6.  

And you got your colors. Of course if you want to join 2 lines, do not inc the color.

And that way you do not need any level from the HL at all.

Eureka!
Wow, @Martin, it helped me a lot. It is the missing part that puzzled me in before. Now I know where the sfaOutlineMergeParent info is useful.
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: x2nie on December 18, 2015, 11:06:03 am
I did, as result, your ideas solves many problem


I introduced many actions as needed. I may add/remove/change later.



Code: Pascal  [Select][+][-]
  1. sfaOutline,  // This node will be higlighted by nested color replacing the token color
  2. sfaOutlineKeepColor, // Direct children should not increase color dept. (But grandchild can.)  e.g. "if","then" any "procedure"
  3. sfaOutlineMergeParent,// This node want to decrease current color depth. (But Previous sibling increased) e.g. "except", "finally"
  4. sfaOutlineForceIndent, // Node will temporary ignore sfaOutlineKeep. (Next sibling can.) e.g in NESTED "procedure"
  5. sfaOutlineHidden,      // Node will not painted by nested-coloring, but may increase color (e.g. any "procedure")    


I have a little bit problem with closing fold tags.
I am still trying  8-)
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: x2nie on December 19, 2015, 02:13:29 pm
Hello!  :)


I am very happy with my last modification.
Finally (after 2 days trying), I can connect each closing-tag's color with their pair opening-tag.
It's only can be done if closing tag has indentical BlockType with opening one.


So let make a deal, PAS (and any other) hl should be redesigned (a little bit modification),
because PAS hl seem as sometime closes fold block with unknown block type.
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on December 19, 2015, 03:58:51 pm
It's only can be done if closing tag has indentical BlockType with opening one.


So let make a deal, PAS (and any other) hl should be redesigned (a little bit modification),
because PAS hl seem as sometime closes fold block with unknown block type.

Example?

This shouldn't be possible. (not if there is valid code).

However closing tags may have 0 size, or NO location:
Code: Pascal  [Select][+][-]
  1. procedure foo,
  2. var
  3.   a: boolean;
  4. begin
  5.  
The var block just ends. there is no keyword for that.
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on December 19, 2015, 05:05:42 pm
Also for this you can use the foldlevel.

if the foldlevel is equal (using it's value only for comparison with other foldlevel) then that is the closing tag. 

If the level is less, then the block was already closed.

EDIT: verify it is in the same fold group too
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: x2nie on December 19, 2015, 07:08:21 pm
It's only can be done if closing tag has indentical BlockType with opening one.


So let make a deal, PAS (and any other) hl should be redesigned (a little bit modification),
because PAS hl seem as sometime closes fold block with unknown block type.
This shouldn't be possible. (not if there is valid code).

However closing tags may have 0 size, or NO location:
Code: Pascal  [Select][+][-]
  1. procedure foo,
  2. var
  3.   a: boolean;
  4. begin
  5.  
The var block just ends. there is no keyword for that.

Even "var" fold block has closing-fold-block. Yeah, the size is maybe zero.
But my current code depends too-much on the very-identical BlockType between open~close tag.


Let's make it clear.
* The original PAS HL (I didn't touch) implements it's own version of InitNode,StartPascalBlock,EndPascalBlock...
so it yet able to fill the correct blocktype.
* But once it needed to obey the changes made in the base class, it's will NOT have opportunity to tell what's the BlockType of closing tag.
It is because the definitions :
StartCodeFoldBlock(p+Pointer(PtrInt(ABlockType)), FoldBlock));
EndCodeFoldBlock(DecreaseLevel);

Okay, if I can't trust the FoldBlockType (because it may difficult to do / too muuch changes needed), i will try to believe in FoldGroup.
Yes, I also worry/confusing about similar block level of inner contents of between {$IFDEF}... & {$ELSE}....
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: x2nie on December 20, 2015, 05:01:20 am
anyway, my markup slowdown when scroll at about 4000 lines.
I test using GR32.pas ( has 6500 lines).
But it fast enough at earlier lines ( < 1000 ) using the same file
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on December 20, 2015, 05:13:11 am
Quote
The original PAS HL (I didn't touch) implements it's own version of InitNode,StartPascalBlock,EndPascalBlock...

Then for now it keeps its own. That can be independently reviewed later.

Quote
anyway, my markup slowdown when scroll at about 4000 lines.
I test using GR32.pas ( has 6500 lines).
But it fast enough at earlier lines ( < 1000 ) using the same file

I might look at it, but later.
I will be travelling. I'll be online but with limited time, and limited access to sources etc.

Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: x2nie on December 21, 2015, 03:58:52 am
anyway, my markup slowdown when scroll at about 4000 lines.
I test using GR32.pas ( has 6500 lines).
But it fast enough at earlier lines ( < 1000 ) using the same file


I got it!
Finding parent + grandpa in TLazSynEditNestedFoldsList, will always continue until the most top the "unit" keyword (root of folds, at first line ) founded !
And it is repeated for each line in my markup class.  :'( (it's not my fault, its default behaviour of NestedFoldLIst).
Anyway, It (find till "unit") is not required by another markup (such as triplet WordGroup markup).


Actually my markup didn't require it (find till "unit") either. Because the first fold to be colored is "procedure".
But naturally TLazSynEditNestedFoldsList will not stop until first  (#zero ) grand parent fold reached.
Yes obviously there are not used grand parents : "unit", "interface", "implementation".
That is why my markup was slowing down: repeat searching the "unit" for each visible line.


---
I have a quick solution: stop search the grand parent when the "procedure" keyword is just reached.
This way, will make my markup only work in Pascal Highlighter, or may need some complex configuration for other HL.


But, you have had another solution: to cache the finding parent in NestedFoldList class.
This way will keep markup independent for any HL.
Using cache will speedup finding root, because of reusing the founded grand parent instead of actual repeated finding.


I think the second (yours) solution is far better.
but,well, my internal problem is TLazSynEditNestedFoldsList just too complex for my brain.
No worry, I am creating my own NestedFold getter class. It will be easier for me when create new class with that everything is under control.
Anther reason is I also hate the ReleaseReference + AddReference (aliens) things.  >:D
It's far better to use Create + Free. Which are common and predicable behavior.



Overall, have a nice day @Martin_fr. Have a nice trip !
I most probably send you pictures rather than ask to check the source code.
Thanks
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on December 21, 2015, 04:01:38 am
Quote
I got it!
Finding parent + grandpa in TLazSynEditNestedFoldsList, will always continue until the most top the "unit" keyword (root of folds, at first line ) founded !

I have several ideas how to fix that. But dont have the time now, so later.
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on December 21, 2015, 04:06:39 am
Quote
I got it!
Finding parent + grandpa in TLazSynEditNestedFoldsList, will always continue until the most top the "unit" keyword (root of folds, at first line ) founded !

I have several ideas how to fix that. But dont have the time now, so later.

edit:
Quote
I have a quick solution: stop search the grand parent when the "procedure" keyword is just reached.
This way, will make my markup only work in Pascal Highlighter, or may need some complex configuration for other HL.

add virtual (pas needs to override to return correct) method to foldblock to return fold-type, then look up fold conf, and look at modes. then you can go up foldblocks to see if you need to scan.

---------
remember last line, and if line increased by one, use the data you have. (EndMarkup to clear cache)

Also if min-level equals last lines end level, then nothing changes on a line.

----------
I rewrite in more detail at some time



----
Quote
Anther reason is I also hate the ReleaseReference + AddReference (aliens) things.  >:D
It's far better to use Create + Free. Which are common and predicable behavior.
compile with assert enabled.

I explain the background on them later
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: x2nie on December 21, 2015, 04:11:37 am
Quote
The original PAS HL (I didn't touch) implements it's own version of InitNode,StartPascalBlock,EndPascalBlock...

Then for now it keeps its own. That can be independently reviewed later.
Yes it was, but Now I changed my mind.
Soon I will directly modify that (PasSynPas.pas).
And for reviewing/comparing reason, the unmodified class will be kept in new file (maybe Original_PasSynPas.pas or PasSynPas1.pas ).


The reason is I want to integrate/test that HL in our Lazarus IDE Editor.
Loading file in IDE Editor is great because of it receives drag/drop file from WindowsExplorer etc.
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: x2nie on December 22, 2015, 02:09:37 am
Hey all, I have a little good news.
My markup is now cross-ifdef-directive aware.
This way, any pascal statement after $else should be in similar level as $ifdef statement level.


For example, DirectChild1 is first nested procedure inside MainProc.
MainProc->begin should be red = #first level
DirectChild1->begin = orange = #second level
GrandChild2->begin = green = #3rd level
SubGrandChild3 = cyan = #4th level


SiblingDirectChild4 = second level = orange
BEGIN of MainProc = #1st level = red


Code: Pascal  [Select][+][-]
  1. {.$define debug_FC_line_changed}
  2. procedure MainProc;
  3. {$ifdef debug_FC_line_changed}
  4. var F : TCustomForm;
  5. begin
  6.   F := GetParentForm(self.SynEdit);
  7.   if F <> nil then begin
  8.     if F <> Form1 then begin
  9.        if F <> Form2 then begin
  10.            if F <> Form3 then begin
  11.                if F <> Form1 then begin
  12.                   Result := Form0; end end end;
  13. {$else}
  14.   function DirectChild1: Integer;
  15.   var i : integer;
  16.     function GrandChild2: Integer;
  17.     var j : integer;
  18.       function SubGrandChild3: Integer;
  19.       var k : integer;
  20.       begin
  21.         repeat
  22.           inc(ANodeIdx);
  23.         until (sfaInvalid in Result.FoldAction);
  24.       end;
  25.     var j2 : integer;
  26.     begin
  27.       Result := SearchLine(YIndex, NIndex);
  28.     end;
  29.   var i2 : integer;
  30.   begin
  31.     Result := -1;
  32.   end;
  33.  
  34.  
  35.   function SiblingDirectChild4( aRow: Integer ): integer;
  36.   var S : string;
  37.   begin
  38.     Result := 4;
  39.   end;
  40.  
  41.  
  42. var y : integer;
  43. begin
  44.   Result := 100;
  45. {$endif}
  46. end;          



Well, although it was almost complete, there are known bug:
It can't resolve this real code:
Code: Pascal  [Select][+][-]
  1. {$IFDEF BCB}
  2. procedure TBitmap32.Draw(const DstRect, SrcRect: TRect; hSrc: Cardinal);
  3. {$ELSE}
  4. procedure TBitmap32.Draw(const DstRect, SrcRect: TRect; hSrc: HDC);
  5. {$ENDIF}
  6. begin
  7.   (FBackend as IDeviceContextSupport).Draw(DstRect, SrcRect, hSrc);
  8. end;      


Any clue?
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: x2nie on December 22, 2015, 02:57:30 am
Wow, everything is fine when we get rid from {$ELSE}. How it can be? I don't know yet.



Code: Pascal  [Select][+][-]
  1. {$IFDEF BCB}
  2. procedure TBitmap32.Draw(const DstRect, SrcRect: TRect; hSrc: Cardinal);
  3. {$ENDIF}
  4.  
  5.  
  6. begin
  7.   (FBackend as IDeviceContextSupport).Draw(DstRect, SrcRect, hSrc);
  8. end;




PS:
Legend:
Kwd: count of available node (keyword) in a line
Nst: count of parent fold in a line
Min: Range.MinimumCodeFoldBlockLevel
End: Range.CodeFoldStackSize
NMi: Range.MinimumNestFoldBlockLevel
Nst: NestFoldStackSize
@#: nested-procedure level
T : =True, Inside procedure Neck (after 'procedure' before 'begin')
.  : =False, =not inside the procedure's neck (maybe inside 'begin' or outside procedure).
R: old sign, obsolete, will be removed in near future.

<- : starting-fold sign
-> : end-of-fold sign


Yeah, I use all of them just for detecting the nested-procedure level. I maybe overkill :)p
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: x2nie on December 23, 2015, 10:59:38 am
@Martin, when code is good enough (used as best practice), I think support them is also best practice (good enough).
So, after several trial, I reach a nice solution.

Well, although it was almost complete, there are known bug:
It can't resolve this real code:
Code: Pascal  [Select][+][-]
  1. {$IFDEF BCB}
  2. procedure TBitmap32.Draw(const DstRect, SrcRect: TRect; hSrc: Cardinal);
  3. {$ELSE}
  4. procedure TBitmap32.Draw(const DstRect, SrcRect: TRect; hSrc: HDC);
  5. {$ENDIF}
  6. begin
  7.   (FBackend as IDeviceContextSupport).Draw(DstRect, SrcRect, hSrc);
  8. end;      


Any clue?
This problem is now solved.


Further, I also solved more complex $IFDEF $ELSE things:
Code: Pascal  [Select][+][-]
  1. {$IFDEF foo}
  2. procedure TBitmap32.Draw(const DstRect, SrcRect: TRect; hSrc: Cardinal);
  3. {$ELSE}
  4.   {$IFDEF bar}
  5. procedure TBitmap32.Draw(const DstRect, SrcRect: TRect; hSrc: Cardinal);
  6.   {$ELSE}
  7.     {$IFDEF abc}
  8. procedure TBitmap32.Draw(const DstRect, SrcRect: TRect; hSrc: Cardinal);
  9.     {$ENDIF}
  10.   {$ENDIF}
  11. {$ENDIF}
  12. begin
  13. end;  


But, I wouldn't solve below code, because it seem ambiguous:
Code: Pascal  [Select][+][-]
  1. {$IFDEF foo}
  2. procedure TBitmap32.Draw(const DstRect, SrcRect: TRect; hSrc: Cardinal);
  3. {$ENDIF}
  4.  
  5.  
  6. {$IFDEF bar}
  7. procedure TBitmap32.Draw(const DstRect, SrcRect: TRect; hSrc: Cardinal);
  8. {$ENDIF}
  9.  
  10.  
  11. {$IFDEF abc}
  12. procedure TBitmap32.Draw(const DstRect, SrcRect: TRect; hSrc: Cardinal);
  13. {$ENDIF}
  14. begin
  15. end;
  16.  

So, no worry, the non-ambiguous logic should be resolved by wrapping $IFDEF altogether, and adding several nested $ELSE like my solution above.
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on December 23, 2015, 02:19:51 pm
I dont know how you did it. But if you use different foldgroups, test with {%region} too.

Also you can not assume any knowledge with foldgroup is what, as that is internal to each HL and can change
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: x2nie on December 23, 2015, 04:46:12 pm
... But if you use different foldgroups, test with {%region} too.
I have no problem with {%region }..{%endregion}, and nested one.
Any other valid & complex code I can test?


edit:
okay, I found the function (about {$ifdef,{$else ) also being called by {%region}
I will make it clear which line is special for $ifdef$else, so it will be easier to understand and maintain.


Quote
I dont know how you did it.
It's easy in theory. (evil is in detail  >:D ) :D


//any nested procedure is counted in each line in InProcLevel. let say: InProcLevel =0.
{$ifdef}
procedure a; //InprocLevel = 1
 function b; //inproclevel =2


// now, inprocLevel = 2
{$else}
// in $else, there are actually 2 node: close-node of $ifdef , and a new open-node of $else it self
// 1) when $else closing the $ifdef (by $else it self), it is a chance to reset InProcLevel to the value of InProcLevel before $ifdef was started
// this time, store InProcLevel to local variable, because the value of InProcLevel will be different whenever EndIfdefLevel changed.
// let say var LocalX := InprocLevel;
// now InProLevel == 0; LocalX == 0;
// 2) when $else has been creating new fold block, InProcLevel reuse last similar EndIfdefLevel value
// now InProcLevel == 2
// ----------------------- the idea is to reset nested procedure level after the $else, to be identic to condition before $ifdef---------------------------
// so, let reuse the local var!
{InProcLevel := LocalX;}
// now InProcLevel == 0;


 procedure a_2; // inproclevel ==1
 procedure b_2; // inproclevel ==2


{$Endif}
// in $endif, we do nothing, just continue the last counter
// now, inProcLevel ==2.
begin end; //inproclevel ==1 // nestd proc
begin end; //inproclevel == 0 // main proc



Quote
Also you can not assume any knowledge with foldgroup is what, as that is internal to each HL and can change
@Martin, in english, please?
what is "is what"? changes what? do you refer to FoldConfig, or refer to change api in future?
who is changer ? do you mean I couldn't change everything I wanted because it is a holy design?
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on December 23, 2015, 05:43:14 pm
I havent sees your code, only will in 2 weeks...

so just generic comments, that may or may not apply.

I dont know how you "solved" ifdef. imho there is no easy way to do so.
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: x2nie on December 23, 2015, 05:54:13 pm
Also you can not assume any knowledge with foldgroup is what, as that is internal to each HL and can change
I try my best to reparse, your intonation might be:
"You can not assume with foldgroup is: as internal to each HL and hence you should not change too much".


Well, martin_fr, for honest I have a few degree of different view of point of "FoldGroup", most probably yes.
I think lazarus (IDE & Editor) has few wrong point of view of FoldGroup too.


Look at Tools>Options > Editor > Display > Markup and Matches (may differ, I use Lazarus 1.5 at home)
it is not really integrated with Toos>Options > Editor > Code Folding.
If/then/else not found in Code Folding, but only appeares in Markup & Matches.
The side effects is it will be complicated when new keyword introduced in fold-config. Let say, I want to introduce folding of 'with~do', 'for~do' etc.

FYI, using my markup class, a 'procedure' can be configured by setting 4 new configuration.
* procedure can completely ignored = exclude(sfaOutline)
* can has/ has not coloring, but maybe has a vertical guideline
* can has / has not vertical outline, but maybe itself has not be colored (like current "then" behavior)
* can visually ignored but has some affect to it's children.
Whatever, how can it be easy to configured?
sure by redesign / redefine the 'fold'. at least I need to redesign the IDE Options's frame.


Another reason: I might change too much, but that was the only way I could do.
It is better for me to change anything than change nothing.

Currently I can't take care of everything beyond SynEdit (such codetool, compiler directive, etc.).
So, if you want to keep IDE in good condition, tell me what shall I do technically.


Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: x2nie on December 23, 2015, 06:12:59 pm
Ouch, did you say "foldGroup" instead "FoldConfig" ? I mean Fold, FoldConfig (folding keywords)

but it still relevant:
How can user configure $ifdef as to be nested-coloring or not?
Some user didn't want to nested color the $ifdef $ifdef $ifdef, but other maybe wanted.
So, still we (I &me ) need to integrated those 2 IDE OptionsFrame (Markup & maches, + Code Folding).


Quote

I havent sees your code, only will in 2 weeks...
so just generic comments, that may or may not apply.


HA HA HAAA.. :P Well I Agree.
But hey, You can not make assumption too, before read the code. Because apply/reject is by strong technical reason.
 >:D 8-)

Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: x2nie on December 31, 2015, 06:51:42 am
Hi,
I have several good news. 8-)


The important news is : I finally able to independently draw markup without FOLD.
it means big change to SynEdit-foldable-Highlighter.
Because lazarus trunk will allow markup to be drawn : only when sfFold (code-folding in IDE) is enabled.


IMHO, it is better: if drawing any markup (such as MarkupWordGroup / paired open~close fold) independently, to being not interfered by code-folding.
in other word (not english) : [sfaFoldFold] <> [sfaMarkup] <> [sfaOutline].


Well, now we can make different between FoldConfig[].Enabled := False; vs FoldConfig[].Modes := -[sfaFold].
This way, "Enabled" will completely affect the whole Modes.
when "sfaFold" will only apply to foldable code, and other modes still active.



another news:
I introduce 3 pascal keywords to be in Markup+Match, Foldable, NestedColoring:

I also deprecating PasFoldLevelEnd, PasMinLevel. >:D  Reusing (and bugfixing) the base class. :-*




Final result: you can disabling fold and in the same time enabling other markup.
see the picture attached.

Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on December 31, 2015, 09:27:50 am
IMHO, it is better: if drawing any markup (such as MarkupWordGroup / paired open~close fold) independently, to being not interfered by code-folding.
in other word (not english) : [sfaFoldFold] <> [sfaMarkup] <> [sfaOutline].


Well, now we can make different between FoldConfig[].Enabled := False; vs FoldConfig[].Modes := -[sfaFold].
This way, "Enabled" will completely affect the whole Modes.
when "sfaFold" will only apply to foldable code, and other modes still active.

absolutely correct. that is how it should be. (And the HL can force some nodes to be enabled, if it needs them for context. e.g. keywords that exists only in certain context)

if there are fixes needed, they should be applied independent of the markup you work on.

Quote
I also deprecating PasFoldLevelEnd, PasMinLevel. >:D  Reusing (and bugfixing) the base class. :-*
that can also be patched on its own.
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: x2nie on December 31, 2015, 10:17:56 am
absolutely correct. that is how it should be. (And the HL can force some nodes to be enabled, if it needs them for context. e.g. keywords that exists only in certain context)
Yeah,
FoldConfig[].enabled, may then only (and still) needed on UnitTest applications.


But, that was not that finish. Lazarus IDE Cod-Folding (editor options) need to be changed too, to reflect the new way.
Anyway, I am trying to little bit redesign those IDE options. in directory: synedit-modified\ide\...
so, I will inform you if I have reached a satisfied result.


Quote
Quote
I also deprecating PasFoldLevelEnd, PasMinLevel. >:D  Reusing (and bugfixing) the base class. :-*
that can also be patched on its own.
I think the modified pascal HL is now stable. And it's baseclass too.
With debug-assert enabled, today I have no error appear.
I also test them with & without my markup. everything seem okay with Lazarus 1.6 and Lazarus 1.7 (a.k.a trunk).


If you have time, you are very welcome to patch them to lazarus trunk :
* SynEdit-modified\SynHighlighterPas.pp
* SynEdit-modified\SynEditHighlighterFoldBase.pas
* SynEdit-modified\SynEditMarkupWordGroup.pp


Quote
if there are fixes needed, they should be applied independent of the markup you work on.
Actually my Markup is also stable enough,
however, I can't recommend to use it now, because : it still slow down when reach lines > 4000.
And my markup is not sexy (even awkward) when 'begin' is not vertically similar to the 'end'.
I have some plan (good ideas) to resolve it later days.


Anyway, I have no further plan / todo with pascal HL. Maybe, just clean up the comments. you can do that.  :-[ :P
But let me know if I can do/correct something with it.
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on December 31, 2015, 10:47:03 am
Quote
FYI, using my markup class, a 'procedure' can be configured by setting 4 new configuration.
* procedure can completely ignored = exclude(sfaOutline)
* can has/ has not coloring, but maybe has a vertical guideline
* can has / has not vertical outline, but maybe itself has not be colored (like current "then" behavior)
* can visually ignored but has some affect to it's children.
Whatever, how can it be easy to configured?
sure by redesign / redefine the 'fold'. at least I need to redesign the IDE Options's frame.

redesign of the ide frame should be no problem. Open to any change needed. But imho thats last on the list.

foldconfig:
I am still thinking about it, there are various options and considerations. Most important, what needs to be known by the HL, and what is markup.

Since the HL needs to deliver the base info for the markup, it also needs some config for this. This means certain config will have to be present, even if the markup is not used at all. (this conflicts with keeping everything modular, but it can't be avoided)

well it could have partly been avoided by adding a collection to foldconfig, and the markup could add a sub classed collection item. But that would cause far to much overhead, so imho not an option.

We first need to look at how many modes are needed.
That means do decide if EVERY configurable keyword can have all options:
- colored and/or lined
- ignored/merged parent or child

merging does not make sense for all the keywords (there in no point in merge "repeat" with parent or child ever).
Imho the HL should have a new property  (enum) to toggle the sensible merges.
Same for ignores.
pasHL.MarkupOutlineOptions = (
moProcedureHiddenIncrease,
moProcedureBeginMerge,
.....
)


as for color vs line:
Just an idea, think about, let me know.

maybe each fold config can have a integer tag.

the markup can then have a list of tags, and properties for it (the problem may be to implement a fast lookup)

e.g.
tags 1,2,3 color
tags 4,5 line
tags 6.7 both
other markups can use the same tags, but group them differently.

If it was only your markup, then we could add 2 modes (outlineColor, outlineLine) and maybe that is the better idea for now.

Only if there are more markups in future this will add many modes.
1) a SET has only so many members
2) the difference between color and line is really none of the HLs business.

Maybe you have more ideas?

btw,tags could also override merge info on the markups side.

------------------
 EDIT
instead of tags  a generic set/enum would also be possible. (more like a bitmap). The HL just copies them from the foldconf, and the markup then can be configured to apply features to individual bits/members
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: x2nie on December 31, 2015, 12:32:41 pm
Quote
FYI, ...
Whatever, how can it be easy to configured?
sure by redesign / redefine the 'fold'. at least I need to redesign the IDE Options's frame.

redesign of the ide frame should be no problem. Open to any change needed. But imho thats last on the list.
Thanks you! 8-)
Quote
foldconfig:
I am still thinking about it, there are various options and considerations. Most important, what needs to be known by the HL, and what is markup.
I keep it simple in mind:
- Markup do know nothing, except the instructions (X,Y, etc.) of drawing.
- HL dictate what should be drawn by Markup, but it always depends on foldconfig.
- HL do know nothing of final/current value of foldconfig. (however, HL provides default foldconfig values.)
- HL knows specific language keyword (repeat,until,begin,while,do, etc.)
Whenever found specific keyword, HL just quickly provides "node" info without seeing what values are delivering from foldconfig.


So, neither HL nor Markup care about foldconfig. It is indirectly from user to user.



It is beautiful design of fold config.
That is why my markup is very simple: modular.! only 500lines. I think I will reduce the codes in future.


Quote
Since the HL needs to deliver the base info for the markup, it also needs some config for this. This means certain config will have to be present, even if the markup is not used at all. (this conflicts with keeping everything modular, but it can't be avoided)well it could have partly been avoided by ...
Hey, there are no conflicting. When HL providing nodes/infos, it is not a big job for HL, because everything has been done once foldconfig changed. And as usually, foldconfigs are not changed during user work with synedit.
Its not a big job for HL : provides the value looked up from foldconfig.
So, I think no worry for this aspect.

Quote
But that would cause far to much overhead, so imho not an option.
I agree, there are no benefit of additional-config-properties of current foldconfig.

Quote
We first need to look at how many modes are needed.
That means do decide if EVERY configurable keyword can have all options:
- colored and/or lined
- ignored/merged parent or child

merging does not make sense for all the keywords (there in no point in merge "repeat" with parent or child ever).
If Merging not applicable for few keyword, let's make it disabled (not configurable) for them.
Look at attachment of how to disabled.

Quote
If it was only your markup, then we could add 2 modes (outlineColor, outlineLine) and maybe that is the better idea for now.
Great! adding fmOutlineColor,fmOutlineLine is the best we can do for now.
These 2 modes, together with fmOutline, can easily be implemented into IDE options. see attachment of how to.

Quote
2) the difference between color and line is really none of the HLs business.
even HL will never care about these, but the HL is the only correct place to put them.
It is the reason of why there are combobox "languange" for configuring them. see attachment, noticeable as green.

Quote
Maybe you have more ideas?
I don't understand what is problem with current foldconfig?
I can't see any benefit of additional tags or enums.
 I can't see any problem with current config's technique.

It is most probably caused by my stupid, I felt I may misunderstand of your ideas.
Can you further talk about these?



edit: +attachment.
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: x2nie on December 31, 2015, 01:11:55 pm
foldconfig:
I am still thinking about it, there are various options and considerations. Most important, what needs to be known by the HL, and what is markup.

Since the HL needs to deliver the base info for the markup, it also needs some config for this. This means certain config will have to be present, even if the markup is not used at all. (this conflicts with keeping everything modular, but it can't be avoided)

well it could have partly been avoided by adding a collection to foldconfig, and the markup could add a sub classed collection item. But that would cause far to much overhead, so imho not an option.


Oops, you said "conflicts" and I did recognize it as it's antonym. sorry.
Now I understand, you might said: adding more foldconfig to HL will add more work to HL, even the markup (who doing the foldconfig job) is not available (disabled). Ideally if markup not available, there are no related work for HL.


And most probably you are worrying about : my markup slowing down when reach line > 4000.


1) Well, about slowing down, it's pure my bug. It would not a bug for other markup class.
You have said that I should introduce a new virtual method that pascal HL can override. I will be solution, but I need more time to implement.


2) About useless foldconfig (because markup not available),
Based on my experience (1 month days and nights), IMHO it's normal.
Additional foldconfig wouldn't make HL work harder. (additional fold config = add more member of a type). And these types are not intensively calculated.
AFAIK, It would not make HL more complicated.
It may only make IDE's editor options a bit more changed.  8-)





Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on December 31, 2015, 01:19:29 pm
Quote
If it was only your markup, then we could add 2 modes (outlineColor, outlineLine) and maybe that is the better idea for now.
Great! adding fmOutlineColor,fmOutlineLine is the best we can do for now.
These 2 modes, together with fmOutline, can easily be implemented into IDE options. see attachment of how to.

I prefer only one mode. Because there may be further markups, or other modules that need to add modes.

then somehow configure on the markup, which nodes to color/line/both.

the markup could have a list of ptrint containing the ord values of cfbt.... that should have color or line.

Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: x2nie on December 31, 2015, 01:37:05 pm
oh, you saying about that.


well, when I did implement exactly what you wanted, then the keywords will be not configurable ([OutlineLine, OutlineColor]) any longer.
Let say I want cfbtForDo to be not colored, how can do that via IDE Editor-Options? We will not able to do that !


As far as I know, IDE editor options will change the HL.FoldConfig[ord(cfbtForDO)].modes := [fmOutline,fmOutlineHasLine, fmOutlineHasColor]
if you prefer only one mode ( [fmOutline] ), how this can be resolved ? >:D
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on December 31, 2015, 02:43:42 pm
markup.drawLines[ord(cfbtForDO)] := true;
markup.drawColor[ord(cfbtForDO)] := true;

of course if you change the HL to xml, then you need to change settings.

on top of that 2 settings

markup.DrawLinesForAll := false // if true then do not check the above is ignored
markup.DrawColorForAll :=
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: x2nie on December 31, 2015, 02:44:17 pm
Now I know the problem. :o
You are keeping HL classes to be not be changed when a new markup class introduced. aren't you?


Modularity = a module no need to be changed when another module introduced.
current situation = all foldable HL need to be changed to reflect of new markup born.


Now, let see what are the constraints:
* user should able to configure HL + Markup behaviors via IDE's editor options.
* HL should not changed too much (even shall not changed) when a new markup introduced.
* Ideally, Markup class is general purpose, not designed for specific programming language.
* FoldConfig is no longer for expand/collapse things only. It became more general purpose.




And what are NOT be constraints:
- FoldConfigs may/ may not attached to HL. it can be independent class, can be looked up using TypeInfo.
- configurations per markup should / should not multiply of available markup.
- known markups may / may not available at a time.
  (But configurations for each markup can be created & destroyed anytime)


So, from above thought this is my idea:
+ Extract the markup config into a new class/record.
Purpose: modularity, keep HL in it's own business.


It should apply to configure MarkupWordGroup, and any newer markup.
This way, new introduced markup would not change any HL.


The final amount of configurable things would be: (HL count * eachHL.configCount) * know markup class.
Pascal.configcount := ord(cfbtIfthen{last}) - ord(cfbtBeginEnd{first})
known markup class := [MarkupWordGroup, MarkupNestedColoring, ..., MarkupNewerClassIntroducedLater]
HL count := [ Pascal, JS, XML, HTML, PHP, ...]


You can see, additional new Markup class will only need additional config, but no change will be needed for each HL.


Note:
1) I don't know whether FoldConfig will be completely detached from HL, because fmFold may still needed by other module (such gutter, TextBuffer, etc.).
2) when a "new" config created, it will not require the markup instance exist,
but we only need the count of them for displaying / playing with them.


It will be similar of registering new type of TGraphic, but that TGRaphic ancestor instance are not required to be exist, just only need to register the class.
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: x2nie on December 31, 2015, 03:10:10 pm

@Martin_fr, it would not work.
markup.drawLines[ord(cfbtForDO)] := true;
markup.drawColor[ord(cfbtForDO)] := true;

of course if you change the HL to xml, then you need to change settings.

on top of that 2 settings

markup.DrawLinesForAll := false // if true then do not check the above is ignored
markup.DrawColorForAll :=
You know, such configurations is needed to be stored some where.
And the current place is in HL.
We cant make markup class as the place to store the configuration.
Can we?
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: x2nie on December 31, 2015, 03:27:22 pm
I have a question:


For each of specific language HL (let say for TPasSynPas), is there possible to have different FoldConfig for multiple instance?


Example: I have two TPasSynPas instance in real life, are there a need to have different FoldConfig ?
PasSynPas1 = if/then/else is marked as triplet markup,
PasSynPas2 = in opposite configuration.
Are these needed in real life?

-=-
if above is possible: fold config shall be inside HL, because different HL instance (same class) should has different config.

if above is Impossible: fold config can be extracted into another class, all similar HL class (multiple instance) has one global config.
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: x2nie on December 31, 2015, 03:34:26 pm
Well, I just changed my mind.


Even there are one global config for all instance (multiple instance same class),
each instance can has a copy of that config.


Meaning no mater inside or ouside HL the config attached,  each instance can has individual config for further make different with other intance (same class).


It also possible to make the config shared among instance or independently, just like your ".AddReference" things !

wow. What's your opinion?
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on December 31, 2015, 03:43:31 pm
You are keeping HL classes to be not be changed when a new markup class introduced. aren't you?
Ideally yes-ish. though its not entirely possible. (at reasonable cost.

Synedit should be modular, and modules should be forced to stay in there boundaries, which improves maintenance. Look at syncro edit, or multi-caret. You can use synedit without them. they are add ons.

Now HL and markup have some dependencies. That is ok. This are dependencies arising from the functionality they should provide. They should not arise from implementation detail.

Though there is a 3rd factor  "usability" (from users point) and that is not always easy to decide.

The dependency here is markup needs HL. But HL does not need markup.

So anything added by the markup should not impose on HL.
"should not". However to some extend it must do so, but should should be minimal.

For the HL it makes no difference, how the outline is painted (line, color-pair, both). That info is not needed to provide matching nodes. So if possible that should be configured on the markup.


Quote
And what are NOT be constraints:
- FoldConfigs may/ may not attached to HL. it can be independent class, can be looked up using TypeInfo.
foldconf class is always defined by the HL (HL subclass)

It may be, that markups can attach additional info

foldconf.foo = TCollection

and a markup could add an inherited TCollectionItem to that. Doing so needs clever lookup, in order to not search the entire collection all the time.

Quote

So, from above thought this is my idea:
+ Extract the markup config into a new class/record.
which part to extract?

Some parts go on the HL and the foldconf (mis-named now), as they are needed by the hl.

Some can go on the markup itself.

Are there currently parts that need to be extracted?

Quote

Note:
1) I don't know whether FoldConfig will be completely detached from HL, because fmFold may still needed by other module (such gutter, TextBuffer, etc.).
2) when a "new" config created, it will not require the markup instance exist,
but we only need the count of them for displaying / playing with them.


fmFold , fmMarkup and (ONE) fmOutline  will stay (as the HL needs it, too provide info).

Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on December 31, 2015, 03:45:50 pm

Even there are one global config for all instance (multiple instance same class),
each instance can has a copy of that config.


Meaning no mater inside or ouside HL the config attached,  each instance can has individual config for further make different with other intance (same class).


It also possible to make the config shared among instance or independently, just like your ".AddReference" things !

wow. What's your opinion?

global meant PER markup, but switching all nodes.

 That is enable/disable per node config.

The module may also be used outside the IDE, so that is helpful for users.

---------
more next year
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: x2nie on January 01, 2016, 03:49:16 am
markup.drawLines[ord(cfbtForDO)] := true;
markup.drawColor[ord(cfbtForDO)] := true;


1) I assume you agree: we put Markup classes  in general purpose level (not for specific programming language) as possible.
MarkupIfDef is excepted. But other markup class should be independent.


2) I can implement what you wanted, so I accepted above code is a concept, it may different in real code.


Now, the problem is : pascal HL has CountPascalCodeFoldBlockOffset that used in too many lines.
This CountPascalCodeFoldBlockOffset are used together with CodeType (cfbtForDo, cfbtIfThen...) using sum operation.
Code: Pascal  [Select][+][-]
  1.  
  2. BlockEnabled := FFoldConfig[ord(ABlockType)].Enabled;
  3.   if (not BlockEnabled) and OnlyEnabled then
  4.     exit(nil);
  5.   FoldBlock := BlockEnabled and (FFoldConfig[ord(ABlockType)].Modes * [fmFold, fmHide] <> []);
  6.   p := 0;
  7.  
  8.  
  9.   if not FoldBlock then
  10.     p := PtrInt(CountPascalCodeFoldBlockOffset);
  11.   Result:=TSynCustomCodeFoldBlock(StartCodeFoldBlock(p+Pointer(PtrInt(ABlockType)), FoldBlock));  
The side effect is : pascal blocktype value sometime is beyond the foldconfig count.
in english: pascal blocktype value is not portable.


It is a bug of pascal HL. keeping this constant will sacrifice other class portability.


My sugested solution is deprecating this CountPascalCodeFoldBlockOffset, using portable logic/value: sfaFold.
You know, CountPascalCodeFoldBlockOffset is just a flag when the block has NOT sfaFold.


Is there another usage of CountPascalCodeFoldBlockOffset that I didn't know?
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on January 01, 2016, 12:16:58 pm
1) I assume you agree: we put Markup classes  in general purpose level (not for specific programming language) as possible.
MarkupIfDef is excepted. But other markup class should be independent.

2) I can implement what you wanted, so I accepted above code is a concept, it may different in real code.
/quote]
Now, the problem is : pascal HL has CountPascalCodeFoldBlockOffset that used in too many lines.
This CountPascalCodeFoldBlockOffset are used together with CodeType (cfbtForDo, cfbtIfThen...) using sum operation.

It is a bug of pascal HL. keeping this constant will sacrifice other class portability.
[/quote]
/quote]
My sugested solution is deprecating this CountPascalCodeFoldBlockOffset, using portable logic/value: sfaFold.
You know, CountPascalCodeFoldBlockOffset is just a flag when the block has NOT sfaFold.


Is there another usage of CountPascalCodeFoldBlockOffset that I didn't know?
[/quote]

I wouldn't call it a bug. It just needs an accessors that returns the true value to the outside world. internally it is allowed to store whatever it wants.

And yes, that may mean it can not make 100% use of the base class.

It may be possible to get rid of it in pas hl, but it is good that it is there as similar usage may arise with any hl  for any reason.

pas hl is more complex than it looks, and the test case only scratches the surface, Making bigger changes may mean having to extend those test.
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: x2nie on January 01, 2016, 01:09:28 pm
Okay, it's not "a bug".
However, too late. I have removed CountPascalCodeFoldBlockOffset together with PasFoldLevelEnd, PasFOldMinLevel.
As replacement, there is a new property to the CodeFoldBlock : Foldable.


THe reason: I want to make Pascal HL more natural. mean it just similar as possible as another HL ability.
We can't make pas HL as that too special, which too hard to follow of creating new HL.




I am in thinking of, maybe we also need to bring PasFoldFixLevel together with LastLineCodeFoldLevelFix,
into base class. (as we did with other PasXxxxLevel properties)
the reason: Python HL (foldable) seem need it.
you know, in Pascal: multiple ansi comment can be hidden (fmHide) together with fmFold,
now in Python: multiple import lines should can be hidden too. AFAiK it need LasLineCodeFoldLevel, isn't it?
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on January 01, 2016, 01:19:25 pm
sorry but it will be a bit until I can review.

And then, it will need to be possible to differentiate between (different) refactor and new functionality. They need to be checked and applied individually.
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: x2nie on January 01, 2016, 01:32:25 pm

Nice. you may in traveling, etc.
If I were able to make svn patch, what are the priorities?


You know, I have a lot of changes.
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: A-M on January 09, 2016, 06:02:12 pm
Hi dear Lazarus developers
i'm so sorry because of my absence
i was completely busy at my work however, i was visiting this topic sometimes

to be honest i'm benefiting from Lazarus at my job so i think i should compensate that and so i'm doing some works for (such as writing a Lazarus handbook)

about this matter (Colorizing SynEdit) as i can see ,dear x2nie you are doing the job as well with the help of maestro Martin  ;)
but if you think my re-entry into the topic can be useful anyway please write a brief description on what did you till now and what i can do (i tried reading last pages but it was not clear for me because i wasn't involved)

With the best regards.
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: x2nie on January 11, 2016, 05:21:26 am
.. please write a brief description on what did you till now and what i can do (i tried reading last pages but it was not clear for me because i wasn't involved)


Sure. Here are what I knew:


* TSynHighligther (aka HL) knows the keywords. it does parsing to know the keywords. So, let it be only who doing parsing.
* TSynEditMarkup (and it's ancestors) is the only correct place of doing additional markup (additional color other than keywords).
* TSynMarkupWordGroup is the simplest markup to learn how HL & markup work together.


* HL shouldn't do drawing it self. never.
* HL provides several signals to be used to draw.
   Such T***Attri for coloring the keyword,comment,symbol by specific colors. It's a legacy from old SynEdit (still founded in Github or Sourceforge).
   and Lazarus's SynEdit provide additional signal via TNodeList.
* TNodeList can be called per-line basis, and can be called several time when needed by markup.
* TNOdeLIst contains several FoldNodeInfo, which each is generally for fold/unfold info (whether it is opening fold or closing one).
* Additionally, each FoldNodeInfo contains set of (
   * sfaMarkup {used by TMarkupWordGroup},
   * sfaOpen,sfaClose {used by TMarkupWordGroup to connect/find paired keyword such begin..end, try/finally/end, procedure/begin/end etc}
   * sfaFold {used by FoldGUtter and CodeBUffer to collapse/expand a block of code, together with sfaOpen&sfaClose}
   * sfaHide {for comments, it is similar as sfaFold, but will collapse to previous line while sfaFold collapse to first-block-line}
   * sfaOutline {it is new, for our colorizing. If any nodeInfo contains "sfaOutline", I got it. If it doesn't contain "sfaOutline" I ignore it}


FoldNodeInfo is generated by  HL, but it depends on Markup class existence to draw.
Well, neither Markup do drawing by itself. Each markup also only provides some instruction of drawing to be done by TSynEdit.
The real drawer is not Markup nor HL. It is because even small individual drawing will cost of time.
Managed drawing will reduce time. Believe it!


Here is a simple diagram of their connectivity:
Form1
  > SynEdit1
        > SynMarkupManager1
              > SynMarkupWordGroup1
              > SynMarkupColororing1
  > SynFreePascalSyn1


some markups has been created automatically inside SynEdit. So, you must add SynMarkupColoring1 manually.


So, what was I really doing?
* The root class of all foldable HL (TSynFoldHighlighterBase) now provide simple sfaOutline for every foldable node.
   But it later depends on ancestor of whether it should foldable or not (and has outline or not) by setting SupportedMode & Mode properties.
* TSynPasSyn (and TSynFreePascalSyn) still never draw any markup by itself.
   It just provide those FoldNodeInfoList which in turn will be used to draw by any markup.
* my TSynMarkupColoring class, for each line, will call HL.NodeInfo[ y ];
   In fact, for each line, TSynMarkupColoring will call that too many time as needed, because I need to get all parent.
 


See? The latest problem which not yet being solved is: it always search all parents, even those parents has been founded for previous line.
Why it is hard? Because it involves some classes, It is not written in a simple function to do that. I need to learn more deeply to achieve that. (and more time)



Don't worry, there is a way for quick jump/  to get be fast involved:


1. search the call of "StartCodeFoldBlock" and "EndCodeFoldBlock" for any foldable HL.
(In TSynPasSyn, these are wrapped in method "StartPascalCodeFoldBlock" and "EndPascalCodeFoldBlock", with different parameter)
Both methods is a chance to provide signals (sfaOpen, sfaFold, sfaHide, sfaOutline, etc.).
So you can detect when it is called by search in whole file (Ctrl+Shift+F).


2. The real nodefoldinfo's generator is method: "DoInitNode". this method is called by above "StartCodeFoldBlock" and "EndCodeFoldBlock".
TSynPasSyn has it's own DoInitNode, because it has 3 group: Neutral/Pascal folding group, {$IFDEF folding group, {%region folding group.
Any other HL seem don't need to override this method.


I highly recomended to learn how TMarkupWordGroup works.
Then you can learn something else, but avoid to learn TPasSynPas too early, because it complexity may confusing you of learning the essential things.


greeting.
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Pascal on March 02, 2016, 09:30:22 pm
Martin_fr, any plans when this will be implemented? I am looking forward using this feature.
Nice work x2nie.
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on March 03, 2016, 12:22:20 am
Last 2 month have been very busy for me, and I still am.

This really is my top 1 priority, as soon as I can make sufficient time.
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Pascal on April 11, 2016, 08:35:59 pm
Hello Martin_fr,

nice to see you working on this! Looks promising so far.

Regards
Pascal
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on April 11, 2016, 09:06:06 pm
Well x2nie did the work.
It was just unfortunate that I got real busy for the start of this year.
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: x2nie on April 12, 2016, 10:01:38 am
Hello, :)


what I can help further?
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: x2nie on April 12, 2016, 06:20:09 pm
Martin_fr, I've tried your modification.
Anyhow, the speed improvement you made is unbelievable : as fast as without MarkupOutLine.
Congratulation 8-)
But, about the visual result of that markup, It was too old, its left far away from my last modification,
Well I believe you did it indeed to control the code quality.





Okay, we have so many improvement beyond that, that you avoid to just apply them at glance.
So now, Let's organize to merge them one by one, since you seem as has time nowadays.


1. I think the good start is to make pas HL more natural (http://forum.lazarus.freepascal.org/index.php/topic,30122.msg197255.html#msg197255).
    It is because people will use pas HL as guidance when they develop their own HL,
    and because pas HL is most comprehensive HL we have.
    In detail,
    a) let we remove the hard to understand "CountPascalCodeFoldBlockOffset ",
        since its not portable/ not reusable, and is actually has another meaning: hidden sfaFold.
    b) let remove InitNode in pas HL, replace with DoInitNode (override virtual).


2. Let's add more happiness to any new HL development, by enabling sfaOutline by-default.
    Using current code, developer should construct the complex fold config when
    they want see the coloring-outline?  it's not fun.
    I think, why being complicated if we can make it easier automatically.
    Once developer ready and satisfied with earlier step, then that is the time to deal with fold config.
    And, it's easy for you too. Just add 2 constants :

Code: Pascal  [Select][+][-]
  1.       act := act + [sfaFold, sfaFoldFold, sfaMarkup, sfaOutline];
from

Code: Pascal  [Select][+][-]
  1.       act := act + [sfaFold, sfaFoldFold];




What do you think? or you may already have another further plan for it now ?

Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on April 12, 2016, 08:24:51 pm
First of all, I am still merging (some of) your changes. Just done another batch.

Done some small fixes and refactor for those.

For pas-hl I could not use the base class CollectNodeInfo, it breaks the test. It also misses the FAtLineStart for hide action, but even if I apply the fix for it (see jscript, at least I think that should fix it), its still fails several tests from the testcase.

It works now, with the CollectNodeInfo being overridden as empty.



I know its still missing for, while, .... on my list.

I will need to carefully look at the other changes you made. More when I get there.



There still is the issue of missing invalidation.

- go to a begin/end block of at least 10 lines.
- position the caret in the line with the "begin",
   but not touching the begin itself.
   Position the caret in the leading spaces of the line.
- now add some spaces.

The begin will indent, so will the horizontal line, but only on the top 3 text lines of the block. The rest will not be redrawn.

Don't try to fix it just now. I will post some more details, of what happens, and what options to fix.







Quote
  a) let we remove the hard to understand "CountPascalCodeFoldBlockOffset ",
        since its not portable/ not reusable, and is actually has another meaning: hidden sfaFold.

Not very urgent at the moment. But yes, eventually will be done. But if I am right, it can be done without adding the FFoldable field.

Currently I do not want to add that field. In fact, I want to reduce FoldLevel and NestLevel into one.

All that can be done, but its a different story. Need to look at the rest first.

Quote
b) let remove InitNode in pas HL, replace with DoInitNode (override virtual).
You already had that done, and I just merged it.

Quote
Let's add more happiness to any new HL development, by enabling sfaOutline by-default.
Let me first finish merging. And see if other bugs turn up that need fixing.

Better to make it avail via Object inspector.

On the long run, SynEdit should tell the HL, if any module needs the info. And the HL can ignore unneeded, yet enable config. I have some ideas there. But that is for when all else is working.
This will be important, if the HL can actually skip work, if some conf is disable or un-needed.

Also the IDE will need to provide options to the user.

Lets first get it working in the IDE.



Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on April 12, 2016, 08:27:29 pm
If I am right, the only outstanding change for merging from your git, are in SynPasSyn?
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on April 12, 2016, 11:31:46 pm
Ok, about the invalidation issue.

First basics.
1) the goal is to invalidate (and then paint) as little as possible. A line should only be invalidated, if something actually changed. (except, currently it is always full lines (but with or without gutter)

2) minimum enclosing rectangle. If more than 1 point is invalidated, the invalidation will cover the minimum enclosing rectangle.
Invalidate line 1, and line 5, and the lines in-between will be repainted too. (but you can not rely on this)

3) Invalidation must be done, before paint.


(2)  is why you may not see the missing invalidation of your vertical line, if other code (word brackets like begin/end) cause invalidation.

(3) means that in GetMarkupForRow and similar, it is to late to invalidate, because those are called only during paint.

(1) means that you need to know the old state, in order to know if the new state is different, and if you need to repaint.


Example:
Code: Pascal  [Select][+][-]
  1. Program a;
  2.    {$mode objfpc}
  3.     begin // some comment
  4.     |
  5.     |  repeat
  6.     |  |
  7.     |  until false;
  8.     end.
  9.  
Lets assume a SynEdit with all other markup disabled, if a line is edited, then only that line is invalidated.

So if the begin was moved, then you need to invalidate all lines down to the end.

Yet if the line with the begin is edited, then you do not know if the begin actually moved, maybe someone edited the comment, or the line starts with a tab, and a space was added before the tab in such way that it did not move anything after the tab.
And if the begin did not move, you are not allowed to invalidate the other lines.

It could also be that the begin or end where deleted. Or changed their level/color as another begin end was added around them.


When changes to the text were made, and before the painting begins, you must find what to invalidate.
You can get the current positions and levels of any needed markup. But you also need the previous, to compare them.

You dont want to store data for 10000 lines of text, only for the current screen. But even then, you can not just go like:
topline = line 1183
bottomline = line 1241
because with folding top and bottomline can be many thousand lines appart.

You want to number the screen lines from 0 to 41 (example). (You then need to listen for resize/zoom, to adjust the space, but more later.)

On each of them you can store the position and
Code: Pascal  [Select][+][-]
  1. 0: [],  //Program a;
  2. 1: [],  //   {$mode objfpc}
  3. 2: [(x: 2, l:1)],  //    begin // some comment
  4. 3: [(x: 2, l:1)],  //    |
  5. 2: [(x: 2, l:1), (x:4, l: 2)],  //    |  repeat
  6.  

That allows you to invalidate correctly.

only, if the screen scrolls, so must your array. Otherwise you would risk to invalidate the entire screen when it scrolls. (And that must not happen)

Also lines may be inserted, or deleted (in text or via fold). One way to detect that is to store with each screen line, what line in the text it is.

If your "program" line is at line 1183 in the text, and scrolled so it's on top of the screen.
Code: Pascal  [Select][+][-]
  1. 0: [1183],  //Program a;
  2. 1: [1184],  //   {$mode objfpc}
  3. 2: [1185, (x: 2, l:1)],  //    begin // some comment
  4. 3: [1186,(x: 2, l:1)],  //    |
  5. 2: [1187,(x: 2, l:1), (x:4, l: 2)],  //    |  repeat
  6.  



A good place to build the list, and do the invalidate is
markup.TextChanged

SynEdit has
    function ScreenRowToRow(ScreenRow: integer; LimitToLines: Boolean = True): integer;
    function RowToScreenRow(PhysicalRow: integer): integer;

to go through all screen lines (including the half visible at the bottom)
Code: Pascal  [Select][+][-]
  1.   for i := 0 to markup.linesInWindow do begin
  2.     realLine := TSynEdit(markup.SynEdit).ScreenRowToRow(i);
  3.     // real line may increase by more than 1, if a folded section is skipped
  4.     ...
  5.   end.
  6.  


---
store the old value of TopLine and LinesInWindow, and check in TextChanged if the values for TopLine or LinesInWindow have changed.

However if no text changed, then TextChanged will not be called. So when GetMarkupFor.... is called you need to check too. In that case invalidation will have been done by whatever caused the change, you only need the data for the current paint

Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Pascal on April 13, 2016, 08:00:55 am
Martin_fr,

your last commit (52184) breaks build. Additional parameter in TSynCustomFoldHighlighter.StartCodeFoldBlock is not added in TIDESynPasSyn.

Regards
Pascal
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: x2nie on April 13, 2016, 11:38:45 am
If I am right, the only outstanding change for merging from your git, are in SynPasSyn?
Apparently (yes).


I've big pain when dealing with pas HL's unit test, and you did it. It helps me a lot when you done with unit-test part.




About invalidation (screen update), you've told me three time (or more).
Previously, you mix it with undo/redo. That's why it so hard for me, because learning undo/redo system is a bit out of topic.
But, today, it is being more clear and clean to understand for me.
Therefor, I will try my best again for this. :-X
Also, this part wouldn't disturb what you are doing.
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: x2nie on April 13, 2016, 12:00:15 pm
Quote
Let's add more happiness to any new HL development, by enabling sfaOutline by-default.
Let me first finish merging. And see if other bugs turn up that need fixing.

Better to make it avail via Object inspector.


Wait ! do you mean that in future, Object Inspector can be used as Code Explorer ? :o
if yes, OI would be very useful for both form and code exploration.
I love this idea :-*
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: x2nie on April 13, 2016, 12:21:04 pm
There still is the issue of missing invalidation.

- go to a begin/end block of at least 10 lines.
- position the caret in the line with the "begin",
   but not touching the begin itself.
   Position the caret in the leading spaces of the line.
- now add some spaces.

The begin will indent, so will the horizontal line, but only on the top 3 text lines of the block. The rest will not be redrawn.


I am dejavu.
Yes you are right I can reproduce what you said,
but I forgot that I did fixed it months ago. I just (forget that I) disabled them.


I've committed,
but please ignore any files except the SynEditMarkupFoldColoring.pas
and please ignore all lines except 1 line:


Code: Pascal  [Select][+][-]
  1. begin
  2.   if EndLine < 0 then exit; //already refreshed by syn
  3.   exit;//debug // <------------ I forgot that I disabled it.
  4.  
  5.  
  6.   y := Caret.LineBytePos.y;


becoming this
Code: Pascal  [Select][+][-]
  1.  
  2.  
  3. begin
  4.   if EndLine < 0 then exit; //already refreshed by syn
  5.  
  6.  
  7.   y := Caret.LineBytePos.y;


Now, it's not an issue any more.
But it's my old changes, maybe you need to review inside the sub procedures.


Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on April 13, 2016, 03:14:13 pm
Martin_fr,
your last commit (52184) breaks build. Additional parameter in TSynCustomFoldHighlighter.StartCodeFoldBlock is not added in TIDESynPasSyn.
Ups forgot that file in the commit, fixed
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on April 13, 2016, 03:17:02 pm
Quote
Let's add more happiness to any new HL development, by enabling sfaOutline by-default.
Better to make it avail via Object inspector.

Wait ! do you mean that in future, Object Inspector can be used as Code Explorer ? :o
if yes, OI would be very useful for both form and code exploration.
I love this idea :-*
No, but OI should be able to show and configure FoldConfig, and markups should be components that can be added like a HL. But that will be some work to get there.
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on April 13, 2016, 05:14:19 pm
On the invalidation. Unfortunately no luck.

You only check, if the change happens in the same line as the caret. But text can change on any line.
Some examples
- indent selection
- unit open in 2 windows (context menu on tab, "clone to new window")
  Edit in one window, and the other window will not update correctly
- Undo/Redo
- syncro edit (changes on potentially dozens of lines)
- multi caret edit, only one caret line, but many carets
- codetools
- editor pascal macros
....

I will be very surprised if you find a solution that does not in any way pre-calculate the columns, and store them. (See my suggestion)
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on April 13, 2016, 07:00:00 pm
Instead of storing data for each screen line, you could also store a list of all the nodes found for the current screen.

If your list is empty (first run), then you have to fill it (in TextCanged), and invalidate all line that will need painting.

the list simply stores a copy of the node from the nested list (and the end line for that node, any other info you may need, such as color level decided)). That has the line, and the x pos.
The list includes the nodes that start off screen, but paint into the screen.

The list does not duplicate:
If line 722 has an opening node that will end in 732, then you do not need to store this node again for line 723...

If lines are folded, you do not search/store for them. But you store such nodes that start in the fold, and reach into the visible.

when lines where changed, then you need to rescan for those lines (if they are visible. (lines that start before visible, will be found by the backward search of the first visible line.
Then you can compare and invalidate.

You can use the content of the list for drawing.

To be able to skip folded lines, you need to work with screen lines.

Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on April 13, 2016, 09:41:01 pm
I had a long look at the remaining changes. (InProcLevel, ...)

As far as I can see, they are dealing with {$ELSE} (and only $ELSE)?

If they have any other effects, without $ELSE (or even without any $IFDEF) then let me know.

Currently I am inclined not to merge them.

They break currently working functionality.
Code: Pascal  [Select][+][-]
  1. procedure foo;
  2. begin
  3. {$IFDEF A}
  4.   try
  5. {$ELSE}
  6.   //
  7. {$ENDIF}
  8.   // ...
  9. {$IFDEF A}
  10.   finally
  11.   end;
  12. {$ELSE}
  13.   //
  14. {$ENDIF}
  15. end;
"try" can no longer be folded.

I have not looked for further examples, but I am sure, I could find them. Equally I am sure that even if you fix this one, there will be others, and some will always be broken.

It is (IMHO) in the nature of ifdef, that (at least without codetools) it is impossible to interpret them correct. If you add code trying, you will simply change one error for another, but pay the price of added complexity.

For example what if instead of $ELSE, the code is {$IFDEF foo} .. {$ENDIF}{$IFDEF opposit_of_foo} .. {$ENDIF}. Or 3 or more consecutive IFDEF, of which you can not detect, if there will be exactly one, zero or one, one or more, zero or more used?

I also think that given the fact that it can always only address a subset of ifdef variations, and that there is no way to predict if users will have those variations or others, the overhead this code adds (in resource usage) is not justified.

-----------------

The only way I can thing that would improve ifdef handling, is ignoring inactive code (like the lowlighting does).

Everything else will be changing one problem for another problem, and which problem is the bigger issue will be each users personal view.

To ignore inactive code codetools is needed. Codetools can not be used inside the HL. It be possible to try a new module to archive ifdef handling (like the lowlighting, or extending that).

But this in definitely not part of this thread. (it is too big a topic of its own)

-----------------


Appart from this a few notes on the implementation.

Those criteria had no part in the decision, any changes to them will not affect the decision.

They are whatever I picked up yet, and probably incomplete

1) It appears that it is limited to 32 nest levels of ifdef, and then what? (32 bits in cardinal)

2) Smart idea to use strings. saving some memory, as copy on write means several ranges can share one string.

2a)
Code: Pascal  [Select][+][-]
  1.    while  L < EndLevelIfDef+1 do begin
  2.       SetLength(DepthProcsMade, L+1);
Set the length before the loop.
This code will reallocate the memory over and over. That can cause major slowdown.

Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on April 13, 2016, 10:19:15 pm
That said, feel free to open the hl ifdef on a separate thread. for discussion what may still be possible.
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Pascal on April 13, 2016, 10:29:37 pm
Martin_fr,

is everything implemented now? It does not seem to work for me as the vertical lines should start at the x offest of the first token of the line.

For example at the if:

Code: Pascal  [Select][+][-]
  1. if condition then begin
  2.   statement;
  3.   if condition then with variable do begin
  4.     if condition then
  5.       statement
  6.     else
  7.       statement;
  8.     statement;
  9.   end;
  10. end else begin
  11.   statements;
  12. end;

Instead they start at the x offset of the token.

Procedure and function should also be colored.

Regards
Pascal
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: x2nie on April 14, 2016, 01:59:17 am
I had a long look at the remaining changes. (InProcLevel, ...)

As far as I can see, they are dealing with {$ELSE} (and only $ELSE)?

If they have any other effects, without $ELSE (or even without any $IFDEF) then let me know.
@Martin:
EndLevelIfDef + InIfdefProcsMade is what you think to deal with {$ifdef}


InProcLevel, InProcNeck is to detect whether the current node is sub-procedure (function,procedure,operator).


There must be reasons of why I introduce several procs/properties (which unfortunatelly I just can't remember them right now).
But you can use the file supplied (demo.pas, demo.js, demo.xml) to visually benchmark once you merge a partial works.
So, whenever you avoid any part of my synpassyn.pas, try your synpassyn.pas to deal/show the demo.pas, you may see the problems.


@Pascal:
yes, what you asked, can only be solved when we reached the next step guided by Martin_fr yesterday:
Instead of storing data for each screen line, you could also store a list of all the nodes found for the current screen.
...
That has the line, and the x pos.
The list includes the nodes that start off screen, but paint into the screen.
The obstacle of previous progress of fixing the vertical line is the speed will be amazingly very slow.
This speed problem will sacrifice the usability, meaning it will make synedit with markup-coloring useless because of the speed degradation.
Today, the speed bottleneck has been fixed by martin, so we can just continue the progress.
(in english: what you requested is indeed in our todo)
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: x2nie on April 14, 2016, 02:11:03 am
@martin:
your last changes can not deal with this:
Code: Pascal  [Select][+][-]
  1. {$IFDEF BCB}
  2. procedure TBitmap32.Draw(const DstRect, SrcRect: TRect; hSrc: Cardinal);
  3. {$ELSE}
  4. procedure TBitmap32.Draw(const DstRect, SrcRect: TRect; hSrc: HDC);
  5. {$ENDIF}
  6. begin
  7.   (FBackend as IDeviceContextSupport).Draw(DstRect, SrcRect, hSrc);
  8. end;
The visual problem: second "procedure" should be drawn as black not as red.
reddish "procedure" indicates that she is sub-procedure, black is first level proc.


Wait, above code is not imaginated, its real code taken from famous gr32.pas.
meaning the code is valid, and there are many similar style of {$ifdef} usage like this.
meaning : you should support the style.


your current change will make all below procedures (of that $ifdef) as red, while actually all above procedure should be black because of they are not sub-proc.
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on April 14, 2016, 04:24:26 am
I will look at your example code. But

@martin:
your last changes can not deal with this:
Code: Pascal  [Select][+][-]
  1. {$IFDEF BCB}
  2. procedure TBitmap32.Draw(const DstRect, SrcRect: TRect; hSrc: Cardinal);
  3. {$ELSE}
  4. procedure TBitmap32.Draw(const DstRect, SrcRect: TRect; hSrc: HDC);
  5. {$ENDIF}
  6. begin
  7.   (FBackend as IDeviceContextSupport).Draw(DstRect, SrcRect, hSrc);
  8. end;
The visual problem: second "procedure" should be drawn as black not as red.
reddish "procedure" indicates that she is sub-procedure, black is first level proc.

Wait, above code is not imaginated, its real code taken from famous gr32.pas.
meaning the code is valid, and there are many similar style of {$ifdef} usage like this.
meaning : you should support the style.

There are many issues with ifdef. from folding, highlight, top-info-line,.... all based on this.

But the issue here is, your fix picks a random constellation in which this happens, works around for that one constellation, leaves the problem for others, and introduces new problems.

An I pointed out in the try-except, some folds stop working.

As for "random" how is your example more important than say
Code: Pascal  [Select][+][-]
  1. {$IFDEF windows}
  2. procedure TBitmap32.Draw(const DstRect, SrcRect: TRect; hSrc: Cardinal);
  3. {$ENDIF}
  4. {$IFDEF linux}
  5. procedure TBitmap32.Draw(const DstRect, SrcRect: TRect; hSrc: HDC);
  6. {$ENDIF}
  7. {$IFDEF darwin}
  8. procedure TBitmap32.Draw(const DstRect, SrcRect: TRect; hSrc: HDC);
  9. {$ENDIF}
  10. begin
  11.   (FBackend as IDeviceContextSupport).Draw(DstRect, SrcRect, hSrc);
  12. end;
Same issue. Only not fixable by any approach inside the highlighter.

If on the other and your argument is that an ifdef/else double procedure declaration is so common, it deserves special handling... Well that can be talked about (maybe, still thinking there may be better ways). But you code goes much further than this, it records ALL nodes in any ifdef. A lot of overhead. (and causing the try except issue).

Also why favour the $else block? In the IDE where either the ifdef or else block will be marked as inactive, this can be very confusing. Imagine the IDE lowlighting the else block (because it is inactive), yet the markup drawing all lines according to the else block. That contraticts itself.

Overhead is an issue too, at least if not optional. I know that your code is fast enough, and memory is cheap.
But here are some cases to consider. Many people are on laptop. If they do not use a feature they do not want their batteries to be drained for it. So if it is just one specific case, then make it just that case. Also make it optional. (e.g. bound to the markup being enabled)

If (and this is a very big IF) the HL ever gets any such code, then it must be very well defined what case(s) exactly it covers, and most important why an exception for that case was made.
So the risk will be kept minimal, that it ends in adding more and more fixes in the HL for something that can not be fixed in the HL.

-------------
Better approach, would be to use info from the lowlighting. That comes from codetools, so it only works in the IDE. And that means it needs a subclassed HL, special for that.
But even that couldn't fix everything.

There may also be a way to solve this outside the HL. But I haven't given it much thought yet.

---------------
About nested procs. you can identify them by checking the TopPascalCodeFoldBlockType, or not? So that needs no extra data to be stored.


Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on April 14, 2016, 04:55:53 am
Here is another example. All levels out correct in the current HL.

Code: Pascal  [Select][+][-]
  1. procedure foo;
  2. begin
  3.   {$ifdef a}
  4.     try
  5.   {$else}
  6.     //
  7.   {$endif}
  8.     //
  9.   {$ifNdef a}
  10.     //
  11.   {$else}
  12.     except
  13.     end;  // wrong procedure end
  14.   {$endif}
  15. end;
  16.  

with your code added. it thinks the procedure ends at the line I commented.

the "try" could also be any other block.

So as I said, you fix one issue by introducing another....
That said, of course there are many thinks that only work with your fix.
But it only proofs you cant highlight all ifdefs correct.
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: x2nie on April 14, 2016, 06:00:39 am
So as I said, you fix one issue by introducing another....
That said, of course there are many thinks that only work with your fix.
But it only proofs you cant highlight all ifdefs correct.
It sad to say, but :-[ apparently (yes) I support a block of pascal syntax and go (I hit and run).
the actual situation:
1. I can't remember in which syntax I introduce them (procs + properties)
2. I didn't able to use unit test that days.


So, sorry. but let me know whenever you passed the unit-test for your code.

Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: x2nie on April 14, 2016, 06:17:59 am

Quote
There may also be a way to solve this outside the HL. But I haven't given it much thought yet.


Better approach, would be to use info from the lowlighting. That comes from codetools,
Yes


Quote
so it only works in the IDE.
No. It should be also work outside IDE, somehow.


Quote
And that means it needs a subclassed HL, special for that.
I think not the HL, but the codetools itself must be subclassed into smaller, which only support $Ifdef.
The idea is to allow user to develop their own pascal syntax editor, with minimum requirements.
The user should be allow to manage list of $ifdef easily.




Quote
But even that couldn't fix everything.
No worry for now, we fix only the known bugs.



Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: x2nie on April 14, 2016, 06:30:40 am
Here is another example. All levels out correct in the current HL.

...

with your code added. it thinks the procedure ends at the line I commented.

the "try" could also be any other block.
Yes, I can reproduce that bug.

and now I remember my reason:
Q:if opening node is inside $ifdef, then which one used ?
A: the last !

{$ifdef foo} for .. do {$else} while...do {$end} ----> the last is "while..do"
{$ifdef baa} try {else} (*don try*) {$end} -----> the last is (*dont try*)
{$ifdef gii} (*raise error here*) {$else} except..end {$end} ----------------- the last is "except..end"
Your code explained:
Code: Pascal  [Select][+][-]
  1. procedure foo;
  2. begin  
  3. {$ifdef a}
  4.     try // -------------- first. will be used if there is no $else  
  5. {$else}
  6.     (*dont try*) //------------------- second. always used as valid node.
  7.          //                                        here the 'first' will be reset as before $ifdef  
  8. {$endif}
  9.     // --- code outside $ifdef block will use the last of both. which in this situation is : '(*dont try*)'
  10.     // note: in current situation, there is no 'TRY', because there is $else that removes it
  11.   {$ifNdef a}  
  12.  //// -------------- first. will be used if there is no $else
  13.   {$else}
  14.       //------------------- second. always used as valid node.
  15.          //                                        here the 'first' will be reset as before $ifdef
  16.     except ///----------------------------- invalid except, so is recognized as identifier rather than reserved word
  17.     end;  // wrong procedure end.   // in this situation, here is no 'try', so it's recognized as paired of 'begin'
  18.   {$endif}
  19. end;
  20.  
As You said, we can't highlight all situation of $ifdef, because we don't know which const is defined in {$ifdef __}.



As about codetools, I think I don't care if we will subclass the HL or CodeTools,
the matter is somehow HL can decide ( by user intervention) of which one of $ifdef is used,
.. not always the last one (as I can only did currently).
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: x2nie on April 14, 2016, 07:20:18 am
---------------
About nested procs. you can identify them by checking the TopPascalCodeFoldBlockType, or not? So that needs no extra data to be stored.
I am trying that approach,


one think I remember is to distinct this 2 different meaning between this:
Code: Pascal  [Select][+][-]
  1.  
  2. Procedure First (n : longint); forward; // <----- NO NEED BEGIN HERE
  3.  
  4.  
  5. Procedure Second;
  6. begin
  7.   WriteLn ('In second. Calling first...');
  8.   First (1);
  9. end;  
  10.  
  11. Procedure First (n : longint); /// REAL BEGIN
  12. begin
  13.   WriteLn ('First received : ',n);
  14. end;      



and this another one:
Code: Pascal  [Select][+][-]
  1. Procedure First ; /// DIRECT BEGIN
  2.   Function Second(P : PChar) : Longint;
  3.   begin
  4.   WriteLn ('In second.');
  5.   end;
  6. begin
  7.   WriteLn ('In First. Calling second...');
  8.   First (1);
  9. end;  
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on April 14, 2016, 02:57:16 pm
Quote
So, sorry. but let me know whenever you passed the unit-test for your code
It passed for all that has been merged.

Quote
Quote from: Martin_fr on Today at 04:24:26 am
Quote
    About nested procs. you can identify them by checking the TopPascalCodeFoldBlockType, or not? So that needs no extra data to be stored.
I am trying that approach,
Please use a branch in your git, based on the pas hl, that is now in svn.

Otherwise I will never be able to merge changes. (Or you need to send patches based on the current svn)

Quote
Code: Pascal  [Select][+][-]
  1.  
  2. Procedure First (n : longint); forward;
This is probably a bug in the current HL. forward should cancel the fold block.
(I might look into that myself, should be easy)

That (and the delay I had on my side) is one of the things that made merging so much work:
Your work contained:
- fixes, that where not just for the markup
- refactor (move to base class), that was not for the markup
- markup changes.

The markup benefits from all of them, but still the first 2 are independent.

Submitting fixes should be separate.

Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on April 15, 2016, 12:35:15 am
Quote
Code: Pascal  [Select][+][-]
  1.  
  2. Procedure First (n : longint); forward;

I just checked. "forward" is already handled. It closes the fold, so the next proc is not affected.
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on April 15, 2016, 03:05:41 am
from your code
Code: Pascal  [Select][+][-]
  1. //avoid bug of IncludeOpeningOnLine := False;
  2.     not ((sfaOpen in TmpNode.FoldAction)  and (TmpNode.LineIndex + 1 = aRow)) then

How to reproduce this bug?
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on April 15, 2016, 03:30:46 am
Code: Pascal  [Select][+][-]
  1.                      sfaOutlineNoColor,     // Node will not painted by nested-coloring, but may increase color (e.g. any "procedure")
  2.                      sfaOutlineNoLine,      // Node doesn't want to have vertical line. (e.g. "then")
  3.  

Should those be in the node?

I would imagine it is better if
Code: Pascal  [Select][+][-]
  1. TSynCustomFoldConfigMode = (fmFold, fmHide, fmMarkup, fmOutline);
would *instead* of fmOutline have
  fmOutlineLine
  fmOutlineColor

Then it can be userconfigured for each node.
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: x2nie on April 15, 2016, 06:11:35 am
@Martin_fr,
unfortunately I am very busy now doing something else (daily job) fyi.
I will back on it perhaps next week. sorry 8)
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: x2nie on April 17, 2016, 08:17:45 am
from your code
Code: Pascal  [Select][+][-]
  1. //avoid bug of IncludeOpeningOnLine := False;
  2.     not ((sfaOpen in TmpNode.FoldAction)  and (TmpNode.LineIndex + 1 = aRow)) then

How to reproduce this bug?
That happened when loading included file {$I including.pas} that has no unit, no program, just direct functions.
But I tried using your last changes, its not a bug.
Try this:



1.
Code: Pascal  [Select][+][-]
  1.  
  2. constructor TSynEditMarkupFoldColors.Create(ASynEdit: TSynEditBase);
  3. begin
  4.   FNestList.IncludeOpeningOnLine := False;//// True; //False; //
  5.  


2.

Code: Pascal  [Select][+][-]
  1.  
  2.     if (sfaOutline in TmpNode.FoldAction )
  3.     //avoid bug of IncludeOpeningOnLine := False;
  4. //    and not ((sfaOpen in TmpNode.FoldAction)  and (TmpNode.LineIndex + 1 = aRow)) then
  5.  


3. open a pascal file that has no "unit", no "interface"/"implementation", not "program".
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on April 17, 2016, 04:28:27 pm
from your code
Code: Pascal  [Select][+][-]
  1. //avoid bug of IncludeOpeningOnLine := False;
  2.     not ((sfaOpen in TmpNode.FoldAction)  and (TmpNode.LineIndex + 1 = aRow)) then

How to reproduce this bug?
That happened when loading included file {$I including.pas} that has no unit, no program, just direct functions.
But I tried using your last changes, its not a bug.

You mean the  procedure(s) at the top, do not get outlined (nor folded)?

This is intention. And important.

And if you plan (which I do not advice) to do some workaround, then make it optional (i.e. provide a boolean property to enable this only if the user wants it.

In such an include file the parser can not know if it is in interface or implementation.
There are many include files that go into the interface part of a unit. All they do is list procedures, that are "exported". So they read like:
Code: Pascal  [Select][+][-]
  1.   procedure foo(..);
  2.   procedure bar(..);
  3.   procedure abc(..);
  4.   .... // many hundred more.
  5.  

I don't think, that in this case you want to start an outline for each of them ?
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: x2nie on April 18, 2016, 07:19:41 am
You mean the  procedure(s) at the top, do not get outlined (nor folded)?
Even worst, there were fatal error such "could not read memory address nyah nyah nya.."
That was the reason I avoid them.
But nowadays, you've changed the classes of nestednodelist, which in turn solves this problem.


Quote
This is intention. And important.
put the situation in unit test maybe enough.
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: x2nie on April 18, 2016, 04:05:10 pm
Code: Pascal  [Select][+][-]
  1.                      sfaOutlineNoColor,     // Node will not painted by nested-coloring, but may increase color (e.g. any "procedure")
  2.                      sfaOutlineNoLine,      // Node doesn't want to have vertical line. (e.g. "then")
  3.  

Should those be in the node?

I would imagine it is better if
Code: Pascal  [Select][+][-]
  1. TSynCustomFoldConfigMode = (fmFold, fmHide, fmMarkup, fmOutline);
would *instead* of fmOutline have
  fmOutlineLine
  fmOutlineColor

Then it can be userconfigured for each node.


How can user configure OutlineLine/OutlineColor if there are no fmOutlineLine/fmOutlineColor?
I do understand the reason, but I can't understand the technical implementation.

Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on April 18, 2016, 05:41:58 pm
How can user configure OutlineLine/OutlineColor if there are no fmOutlineLine/fmOutlineColor?
I do understand the reason, but I can't understand the technical implementation.

When you retrieve the node from the foldednestlist, then it contains the FoldType. (e.g. ord(cbftTryExcept)).
Your markup code can then get the foldconfig for this FoldType, and check the FoldConfig.Modes. That way you know if to color the keyword and/or outline.
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: x2nie on April 19, 2016, 04:17:34 am

Oh! so...
Now I completely agree that the markup should be TComponent.
... OI should be able to show and configure FoldConfig, and markups should be components that can be added like a HL. But that will be some work to get there.
I try this : componentize the markup
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on April 19, 2016, 04:43:21 am
But please keep it in a separate branch.

I don't want to separate the changes again.
In order to merge your code, the outlining, and the component-izing are 2 entirely separate and unrelated changes. And if they are mixed in one branch that creates lots of extra work for me.

--
Also I may have a couple of ideas, but I will write them down later.
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on April 19, 2016, 03:45:48 pm
There may be several ways to make the Markup designer ready.
(Note: publishing required HL properties, is a different issue/branch again)

1) just make them components.
This will also make the MarkupMgr a component, but that one must not be registered.

Also Markups then need a publisher read/write SynEdit property, but the MarkupMgr MUST NOT have that.

A Markup then is kind of owned by the SynEdit and/or the form.
Owner = form, but that means form is responsible for destroying, yet old code assumes SynEdit is, and that also needs to be kept, for compatibility. (SynEdit needs to check, if owner = form)

2) Make SynEdit the owner even in design. (like frames).
SynEdit afaik already has the attributes. It just needs to accept the markup and drop (and forms must reject it).
Investigate gutter parts. they are owned by the SynEdit. (look at an lfm)

3) Similar to 2, but do not add Markup to the component palette.
Instead have a special property editor. (like gutterparts.)

--------
I prefer 2 and 3 over 1.

Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: x2nie on April 20, 2016, 03:41:09 am
okay, I sum up: how the markup owned by synedit is just similar mechanism as
TAction (and TCutAction, TPasteAction, TBoldAction) owned by TActionManager.
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on April 20, 2016, 04:15:41 am
I havent looked at TAciton.

But I was thinking more like any TComponent on a TFrame. When streamed, they are nested into the frame, instead of the form.

SynEdit already does this with Gutter parts.

---
Btw arent we getting ahead of ourselves.

Shouldnt the markup first be finished, before starting to add new features?

Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Pascal on April 20, 2016, 08:58:52 am
Shouldnt the markup first be finished, before starting to add new features?

Yes, of course! I am looking forward of seeing this implemented.  ;)
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on April 20, 2016, 02:53:01 pm
There may be several ways to make the Markup designer ready.

Further more: (Mainly issues with 2 or 3)

Current SynEdit has many markup pre-installed. Ideally (if they are unmodified) they should not be streamed.

Alternatively, a 2nd SynEdit without any markup pre-installed could be created and published. But then it is not only markup, there are "views" (folded view), and maybe other things. And its all or none.

Also, when markups are editable via OI, then some of them are already editable via properties on SynEdit, and that must not cause conflicts.

And lastly if you can add/remove them, markups must be ignored if they are added by other modules (sync edit does add markups)
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: llutti on May 31, 2016, 09:32:06 pm
Hi,

  I download the SVN version of synedit and I tried to use the new TSynEditMarkupFoldColor, but when I run my application occurs an error in line 1551 of the file SynEditHighlighter.pp in function StartAtLineIndex when using the procedure SetLine.

  My question is, can I using this new markup to test?

Regards,

Luciano
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Pascal on June 21, 2016, 02:54:06 pm
Martin, x2nie,

i've build a patch to make it work again in the ide and
did an update for the position of the vertical line.

Please review and commit it to trunk.

Regards
Pascal
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on June 21, 2016, 04:04:26 pm
Please put on mantis, so it will not be forgotten.

What is FColumnOfFirstCharacterInRow? It is never set. (Other than initial -1)

Also this isn't exactly something that belongs in the HL. The hl should not be burdened with added work.

Please keep different fixes and/or optimizations separate.
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Pascal on June 21, 2016, 04:17:38 pm
It's to get the first non space charachter for the line.
In this column the vertical line will be.
It's set in the Next procedure of the highlighter (only implemented in pas right now).
So the Markup does not have to scan every line again and again.
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on June 21, 2016, 04:33:16 pm
I assumed that much.

But the pas hl does not appear to be in the patch?

Also it should be once per line, never mind where. And the HL is used without markup too, so any burden that can be avoided, should be avoided.
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Pascal on June 21, 2016, 05:07:16 pm
You are right. It's missing.
I've attached an updated patch.
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on June 21, 2016, 07:46:37 pm
Again:

1) bugtracker, otherwise it gets forgotten. (sorry it may be a while until I have time to look at it)

2) separate actual fixes, from enhancements.
As it currently stands FColumnOfFirstCharacterInRow is not going to make it.
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Pascal on June 22, 2016, 09:17:45 am
Okay.

I will redesign it and submit a patch to mantis.
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Pascal on July 19, 2016, 11:19:57 am
Martin,

2 questions:

How to handle the kept color if "for/do", "with/do" and "if" start in different rows (green lines)? I think in this case the color should not be kept.

What about the vertical line at the "and"s? Should it be there or should it be omitted?

Regards
Pascal
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Pascal on July 19, 2016, 11:32:02 am
Martin,

the "else" part should also have a verticalline like the "if" part, shouldn't it?
How can i get the information about the "else" part? There is not TPascalCodeFoldBlockType for "else".
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on July 19, 2016, 01:48:16 pm
The code is currently as contributed by x2nie.

I havent looked at it yet in detail, so I dont know the answers.

I have been very busy so far for this year, and I dont know when there will be more work on this.
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Pascal on July 19, 2016, 03:29:40 pm
And what is your opinion to the post before?
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on July 19, 2016, 04:07:49 pm
There will be user config which keywords get a line, and which do not. I already have some code for that, but it is not committed.
There are actual painting bugs in the code at the moment (search this thread for "invalidation"), so until they are fixed, I am not adding anything else.

Also if you plan to make changes to the code, keep in mind, that the invalidation issue may mean, that substantial parts may need to be written from scratch. So changes may get lost....

I have not spent much thought on, if for/do/with should be one or 2 colors (personally, 2 colors, or only ONE line at all), but maybe configurable.

As for the "and" that is the "if" line. Lines (like folding) start at the keyword, maybe the better choice of keyword would be the "then"
But that is fine tuning. (maybe even already configurable)

Yes to the "else".

Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Pascal on July 27, 2016, 03:00:45 pm
Martin,

i've managed to only "sfaOutlineKeepColor" if child is on the same line.
As cfbtProcedure also has sfaOutlineKeepColor set, which always has to keep color
i introduced a "sfaOutlineKeepColorOnSameLine" for if/then, fo/do, while/do and with/do.

Code: Pascal  [Select][+][-]
  1.     if (PasBlockType in [cfbtIfThen,cfbtForDo,cfbtWhileDo,cfbtWithDo]) then
  2.       Include( aActions, sfaOutlineKeepLevelOnSameLine);
  3.  
  4.     if (PasBlockType in [cfbtProcedure]) then
  5.       aActions := aActions + [sfaOutlineKeepLevel,sfaOutlineNoColor];

For the else part of the if-statement:
Is ist posible to implement sfaOpen/sfaClose in the Pascal highlighter?
Otherwise i can not have a vertical line for the else part.

Regards
Pascal
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on July 27, 2016, 03:31:54 pm
the else will need a foldblock, but only if needed by config.

Also I am not sure if it should uset the cfbtIfThen, or need a new one.
Both solutions might have side effects.
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Pascal on July 27, 2016, 10:48:36 pm
Martin,

i've upgraded TSynPasSyn to use foldblock for "else". I've added cfbtIfElse.
I willl add patch to mantis after further testing.

I also will cleanup TSynEditMarkupFoldColors and have a look at the invalidating part.
It's working most of the time as expected already. Also patch to mantis afterwards.

Regards
Pascal
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Pascal on July 28, 2016, 11:09:55 am
Martin,

is it possible to add the line index of the sfaClose-node to TSynFoldNodeInfo of the sfaOpen-nodes?
This would make invalidation much easier.

Pascal
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on July 28, 2016, 01:34:30 pm
is it possible to add the line index of the sfaClose-node to TSynFoldNodeInfo of the sfaOpen-nodes?
This would make invalidation much easier.

This would mean that when scanning for sfaOpen node, the HL always must search for the close line. Even if calling code does not need this. (if there are 10 open nodes on a line, and only one is needed....)

So it should not be added.

-------------
TLazSynEditNestedFoldsList  should provide this already (and more efficient)

Probably you should get the nodes ONLY via TLazSynEditNestedFoldsList. And then you have all info you need.

Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Pascal on July 28, 2016, 02:15:21 pm
This would mean that when scanning for sfaOpen node, the HL always must search for the close line. Even if calling code does not need this. (if there are 10 open nodes on a line, and only one is needed....)

So it should not be added.

But the highlighter keeps a stack of the BlockTypes (TopCodeFoldBlockType). Wouldn't it be easy to just add the nodeinfo of the sfaOpen-node to this stack? On sfaClose you can update the node with the actual lineindex as closing lineindex.

Unfortunately i do not realy understand how these nodeinfos are handled by fold-highlighters. Is there a wiki/doc for this?
I would also like to implement folding for my COBOL-highlighter.

TLazSynEditNestedFoldsList  should provide this already (and more efficient)

Probably you should get the nodes ONLY via TLazSynEditNestedFoldsList. And then you have all info you need.

This would have been my plan B ;)
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on July 28, 2016, 06:37:54 pm
http://wiki.lazarus.freepascal.org/SynEdit_Highlighter

That is not how the "range" info (and the "stack" that is part of the range) work.

1) They are re-used. one "range" can be pointed to by 100 different lines (even in diff files).

2) Even if the endline could be stored, currently if an empty line (or line without fold relevant info) is inserted, only that line is scanned (well that and the next).
If endline info was stored, any inserted line would need to find all surrounding nodes, and update them.
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Pascal on July 29, 2016, 05:43:09 am
ah, okay. Then it will be plan B. :D
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on July 31, 2016, 08:57:27 pm
is it possible to add the line index of the sfaClose-node to TSynFoldNodeInfo of the sfaOpen-nodes?
This would make invalidation much easier.

In the meantime I see why that question.

I though TLazSynEditNestedFoldsList had a function to get the endline.

But it is on TSynCustomFoldHighlighter: FoldLineLength on FoldEndLine
They can find the line much faster.

TLazSynEditNestedFoldsList should probably get a method to get the end-line, based on the index of he node:
TLazSynEditNestedFoldsList.NodeEndLine[Index: integer]
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Pascal on August 01, 2016, 01:54:27 pm
Thanks,

works. See latest patch on mantis.
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: x2nie on August 06, 2016, 03:28:27 pm
See latest patch on mantis.


Which one?
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Pascal on August 07, 2016, 02:39:58 pm
File http://bugs.freepascal.org/file_download.php?file_id=24861&type=bug (http://bugs.freepascal.org/file_download.php?file_id=24861&type=bug)
of http://bugs.freepascal.org/view.php?id=30421 (http://bugs.freepascal.org/view.php?id=30421)
It's a patch to the trunk version.
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Pascal on August 15, 2016, 09:30:25 am
There is a bug in last patch. Please use http://bugs.freepascal.org/file_download.php?file_id=24910&type=bug (http://bugs.freepascal.org/file_download.php?file_id=24910&type=bug)
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on August 15, 2016, 10:23:01 am
FYI: I'll be away the next couple of weeks.
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Pascal on August 15, 2016, 12:03:36 pm
FYI: I'll be away the next couple of weeks.
Holidays or business travel? Anyway, have a good journey!
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Pascal on August 15, 2016, 10:23:01 pm
x2nie,

did you test the patch?
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Pascal on August 16, 2016, 08:29:15 am
It seems that Highlighter.EndLine() does not work correct for "for" "while" "if" "do".
If theese keywords are alone on a line Endline() is 0!
TSynFoldNodeInfo.FoldLvlStart and FoldLvlEnd are always the same for this lines. FoldLvlEnd should be one higher than FoldLvlStart!
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Pascal on August 16, 2016, 09:09:38 am
Code: Pascal  [Select][+][-]
  1. function TSynPasSyn.FoldEndLine(ALineIndex, FoldIndex: Integer): integer;
  2. var
  3.   lvl, cnt, atype : Integer;
  4.   node: TSynFoldNodeInfo;
  5. begin
  6.   node := FoldNodeInfo[ALineIndex].NodeInfoEx(FoldIndex, [sfaOpenFold, sfaFold]);
  7.   if sfaInvalid in node.FoldAction then exit(-1); <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< This causes EndLine to be 0 (ToPos(-1) = 0)
  8.   if sfaOneLineOpen in node.FoldAction then exit(ALineIndex);
  9.   case TPascalCodeFoldBlockType(PtrUInt(node.FoldType)) of
  10.     cfbtRegion:
  11.       atype := 2;
  12.     cfbtIfDef:
  13.       atype := 3;
  14.    .
  15.    .
  16.    .                                                              

EndLine does not work for nodes which have no real folding.

I added a patch to use the endline of the next outer node with real folding until this is fixed by martin_fr.
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: x2nie on September 02, 2016, 09:09:29 am
x2nie,

did you test the patch?


I have had big problem running my app demo that uses synedit+SynEditMarkupFoldColoring.pas, something has been changed somewhere in months.
But I found the bugfix. The problem is in my demo app (that earlier works):


Code: Pascal  [Select][+][-]
  1. procedure TForm1.FormCreate(Sender: TObject);
  2. var
  3.    M : TSynEditMarkupFoldColors;
  4.    S : TSynEdit;
  5. begin
  6.    s := SynEdit1;
  7.    s.Highlighter := nil;   // my original markupcoloring unit doesn't require this
  8.    M := TSynEditMarkupFoldColors.Create(s);
  9.    s.MarkupManager.AddMarkUp(M);
  10.    s.Highlighter := SynFreePascalSyn1; // so this reseting doesn't required by my original markupcoloring unit.
  11. end;
  12.  




Interesting to see in SynEditMarkupFoldColoring, that the error occurred from this line:

  NestCount := FNestList.Count;


that forwarding to:

function TLazSynEditNestedFoldsList.Count: Integer;
begin
  if (FCount < 0) then begin
    InitCount;
  end;
  if FIncludeOpeningOnLine and (FOpeningOnLineCount < 0) then begin
    InitOpeningOnLine;
  end;


  Result := FCount + OpeningOnLineCount;
end;


and finally this triggering error on:

procedure TLazSynEditNestedFoldsList.InitCount;
begin
  if FHighLighter = nil then exit;
  FHighLighter.CurrentLines := FLines;


  FCount := FHighlighter.FoldBlockEndLevel(FLine - 1, FFoldGroup, FFoldFlags); <-------------------- Why it is needed on InitCount ????
  FEvaluationIndex := FCount;
  SetLength(FNestInfo, FCount+1);
end;

which is end up with this line:
Code: Pascal  [Select][+][-]
  1. function TSynPasSyn.FoldBlockEndLevel(ALineIndex: TLineIdx;
  2.   const AFilter: TSynFoldBlockFilter): integer;
  3. var
  4.   inf: TSynPasRangeInfo;
  5.   r, r2: Pointer;
  6. begin
  7.   Assert(CurrentRanges <> nil, 'TSynCustomFoldHighlighter.FoldBlockEndLevel requires CurrentRanges');      
  8.  

However, we can make next progress.
Thanks @Pascal !
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Pascal on September 02, 2016, 03:15:16 pm
I have had big problem running my app demo that uses synedit+SynEditMarkupFoldColoring.pas, something has been changed somewhere in months.
But I found the bugfix. The problem is in my demo app (that earlier works):


Code: Pascal  [Select][+][-]
  1. procedure TForm1.FormCreate(Sender: TObject);
  2. var
  3.    M : TSynEditMarkupFoldColors;
  4.    S : TSynEdit;
  5. begin
  6.    s := SynEdit1;
  7.    s.Highlighter := nil;   // my original markupcoloring unit doesn't require this
  8.    M := TSynEditMarkupFoldColors.Create(s);
  9.    s.MarkupManager.AddMarkUp(M);
  10.    s.Highlighter := SynFreePascalSyn1; // so this reseting doesn't required by my original markupcoloring unit.
  11. end;
  12.  

I will investigate this. Should work without setting highlighter to nil and back to SynFreePascalSyn1.

Thanks @Pascal !

You are welcome!
Was just self-interest!
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Pascal on September 06, 2016, 05:51:40 am
I will investigate this. Should work without setting highlighter to nil and back to SynFreePascalSyn1.
I found the problem: SetLines does not recreate FNestList.
Fixed. Try latest patch.
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on September 10, 2016, 02:33:20 pm

I have had big problem running my app demo that uses synedit+SynEditMarkupFoldColoring.pas, something has been changed somewhere in months.
But I found the bugfix. The problem is in my demo app (that earlier works):


Code: Pascal  [Select][+][-]
  1. procedure TForm1.FormCreate(Sender: TObject);
  2. var
  3.    M : TSynEditMarkupFoldColors;
  4.    S : TSynEdit;
  5. begin
  6.    s := SynEdit1;
  7.    s.Highlighter := nil;   // my original markupcoloring unit doesn't require this
  8.    M := TSynEditMarkupFoldColors.Create(s);
  9.    s.MarkupManager.AddMarkUp(M);
  10.    s.Highlighter := SynFreePascalSyn1; // so this reseting doesn't required by my original markupcoloring unit.
  11. end;
  12.  


Interesting to see in SynEditMarkupFoldColoring, that the error occurred from this line:

  NestCount := FNestList.Count;

From a quick look (not sure if your version, or some patched / older), I can not see where the HL for nestlist if updated, if it changed in synedit.

---
EDIT: I see Pascal found it...
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Pascal on September 12, 2016, 09:11:14 am
Martin_fr, did you read reply #240 and #241?
TSynPasSyn.FoldEndLine() does not work for "while" "do" "if" "for".
Is there a chance to get it work? At the moment i will use the next outer fold which has real folding to find
the endline for invalidation.
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on September 15, 2016, 10:22:17 pm
Do you test with
Code: Pascal  [Select][+][-]
  1. -Criot -gt -gh  -dSynAssert  -dSynAssertFold
  2.  
Because I get a lot of range check errors...

I added code for finding the end line.
Though I only tested with normal folds, not with if/for/do/...
But it should work for them to.

TLazSynEditNestedFoldsList has now a property NodeEndLine
Code: Pascal  [Select][+][-]
  1. NestList.NodeEndLine[i]
should give you the correct (0 based) endline.
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Pascal on September 16, 2016, 12:10:02 pm
Do you test with
Code: Pascal  [Select][+][-]
  1. -Criot -gt -gh  -dSynAssert  -dSynAssertFold
  2.  
No, i didn't before. But with this options i do not get any errors, neither in my testprogramm nor in the ide.
Did you use the latest patch?
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on September 16, 2016, 12:51:27 pm
patch_v10

Maybe it is, because I used the IDE config (see my patch), and I outline only some, but not all keywords.

I will investigate that later.
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Pascal on September 16, 2016, 12:55:14 pm
I also use your patch for Outline-Config
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Pascal on September 16, 2016, 01:03:05 pm
If i disable some keyword for outlining i also get errors.
How can i configure some keywords to not be outlined in a test programm?
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on September 16, 2016, 01:17:27 pm
If i disable some keyword for outlining i also get errors.
How can i configure some keywords to not be outlined in a test programm?

Code: Pascal  [Select][+][-]
  1.   for i := 0 to HL.FoldConfigCount-1 do begin
  2.     HL.FoldConfig[i].Enabled := ...;
  3.     HL.FoldConfig[i].Modes := ... [fmMarkup, fmFold, fmHide, fmOutline]; // or  subsets..... (some modes may not work on some conf)
  4.   end;
  5.   // or HL.FoldConfig[ord(cfbtBeginEnd)]
  6.  
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Pascal on September 16, 2016, 01:27:36 pm
I'vefound the bug.

It seems that DoTextChanged gets called with StartLine and Endline as indeces when configuration is saved.
I've added a workaround http://bugs.freepascal.org/file_download.php?file_id=25036&type=bug (http://bugs.freepascal.org/file_download.php?file_id=25036&type=bug)

By the way: If i disable "Outline (global)" i can not set "Outline" for the individual keyword as it is disabled.
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Pascal on September 16, 2016, 01:43:19 pm
The cause was a wrong call of TextChanged with a StartLine of 0 which is wrong as StartLine is not meant to be an index.

See attached patch.

So you can forget about patch 11  :D
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on September 16, 2016, 02:47:17 pm
please report as a new bug.
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Pascal on September 16, 2016, 02:52:28 pm
Done http://bugs.freepascal.org/view.php?id=30605 (http://bugs.freepascal.org/view.php?id=30605)
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on September 16, 2016, 08:35:42 pm
Applied.

still getting range checks.
- Disable global outline
- Edit the line from your patch in SynEdit
Code: Pascal  [Select][+][-]
  1. fMarkupManager.TextChanged(1 [<< edit/backspace here] , FTheLinesView.Count, 0);
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on September 17, 2016, 01:23:30 am
OK, I found one reason for the crashes.

If I have one unit open in more than one window.
In the 2nd window (SynEdit.ShareTextBufferFrom) the FEndLine array (and other) a zero length.

Because this changes the internal textbuffer (not its content), it does not sent the usual signals.
But it also does not trigger SetLines, because the Markup uses a wrapper version of the textbuffer, and that does not change.

It probably should send lines changed. But I need to see, if that has side effects.
(It needs some more refactor....)

-----------------------
Why does FEndLine has the same length as SynEdit.Lines (If I read that correct)?

You only need to know about current displayed lines. (then the above problem would not exist).

If I am correct you keep an array with length of 10000 (for some units) where you may need only 50 (on screen).

(Well that will become a problem, if we do an overview-panel, showing the whole unit, with font-size = 1, but then not sure if the markup should be applied there.)
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Pascal on September 17, 2016, 06:05:19 am
Why does FEndLine has the same length as SynEdit.Lines (If I read that correct)?

You only need to know about current displayed lines. (then the above problem would not exist).

If I am correct you keep an array with length of 10000 (for some units) where you may need only 50 (on screen).
Yes the array is as long as the SynEdit.Lines. It's a chache for the endline for that line. The same with FFirstCharacterColumn as cache for the
column of the first character in the line (this is the position of the vertical line).

I can not limit this to the lines in the window as i have to at least keep track of FFirstCharacterColumn as i have to draw vertical lines for nodes
that where opened before Topline or i have to refind the positions for every invalidation but we would like to save cpu/energy if i got you right.

I also found an other bugs: Disable outling, select a complete class definition and apply a conditional define (shift + alt + d) ->
Code: [Select]
TApplication.HandleException Arithmetic overflow
  Stack trace:
  $00B2DAE3  TLAZSYNTEXTAREA__INVALIDATELINES,  line 1299 of lazsyntextarea.pp
  $00B2CF3E  TLAZSYNSURFACEMANAGER__INVALIDATETEXTLINES,  line 1117 of lazsyntextarea.pp
  $00C3609C  TSOURCELAZSYNSURFACEMANAGER__INVALIDATETEXTLINES,  line 1366 of sourcesyneditor.pas
  $007325F2  TCUSTOMSYNEDIT__INVALIDATELINES,  line 2819 of synedit.pp
  $00AE346C  TSYNEDITMARKUP__INVALIDATESYNLINES,  line 337 of syneditmarkup.pp
  $00BD1E03  TSYNEDITMARKUPFOLDCOLORS__DOTEXTCHANGED,  line 706 of syneditmarkupfoldcoloring.pas
  $00AE3B1E  TSYNEDITMARKUP__TEXTCHANGED,  line 442 of syneditmarkup.pp
  $00AE46D9  TSYNEDITMARKUPMANAGER__TEXTCHANGED,  line 614 of syneditmarkup.pp
  $00739F11  TCUSTOMSYNEDIT__SCANRANGES,  line 5080 of synedit.pp
  $007311DD  TCUSTOMSYNEDIT__DODECPAINTLOCK,  line 2381 of synedit.pp
  $004D606D  TMETHODLIST__CALLNOTIFYEVENTS,  line 307 of lazmethodlist.pas
  $00AFE97D  TSYNEDITSTRINGLIST__SENDNOTIFICATION,  line 1545 of synedittextbuffer.pp
  $00AFCAAF  TSYNEDITSTRINGLIST__SETUPDATESTATE,  line 1248 of synedittextbuffer.pp
  $00AF64BE  TSYNEDITSTRINGS__ENDUPDATE,  line 973 of lazsynedittext.pas
  $00730CF7  TCUSTOMSYNEDIT__DECPAINTLOCK,  line 2305 of synedit.pp
  $00742213  TCUSTOMSYNEDIT__INTERNALENDUNDOBLOCK,  line 7148 of synedit.pp
  $00738577  TCUSTOMSYNEDIT__SETSELTEXTEXTERNAL,  line 4586 of synedit.pp
I will investigate this later.
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Pascal on September 17, 2016, 07:42:21 am
Fixed! Added new patch wich also uses NestList.NodeEndLine.
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on September 17, 2016, 11:31:03 am
Yes the array is as long as the SynEdit.Lines. It's a chache for the endline for that line. The same with FFirstCharacterColumn as cache for the
column of the first character in the line (this is the position of the vertical line).

I can not limit this to the lines in the window as i have to at least keep track of FFirstCharacterColumn as i have to draw vertical lines for nodes
that where opened before Topline or i have to refind the positions for every invalidation but we would like to save cpu/energy if i got you right.

All you need is a 2nd array (for each of the two), that contains all the node-infos opened before topline.

Code: Pascal  [Select][+][-]
  1. procedure Foo();
  2. begin
  3.   if Message.WParam = WParam(FGroupIndex) then
  4.   begin
  5.     Sender := TCustomSpeedButton(Message.LParam);
  6.     if Sender <> Self then
  7.     begin
  8.  
  9. ------------------- Topline
  10.       if Sender.Down and FDown then
  11.       begin
  12.         FDown := False;
  13.  

FIrstBeforeTopline = (1, 3, 5); // does not matter on which line they are, just that they exist
First = (0 {visible line 1, nothing new opening // or is it 5 repeated?}, 7, 0 ....  ); 

Of course you need to adapt things on scroll.... you need to know which line topline was before.

---------------------
If you must keep full length, then use the managed ranges. (one class can hold both: first and len)

See FTabDAta in TSynEditStringTabExpander.TextBufferChanged

This well automatically be kept always the same length as Lines, and if insert/delete happens this is automatically done in the right place (you only need to zero/initialize it.


Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Pascal on September 19, 2016, 12:17:38 pm
Martin,

i added a new patch http://bugs.freepascal.org/file_download.php?file_id=25046&type=bug (http://bugs.freepascal.org/file_download.php?file_id=25046&type=bug).

This solved the issue with cloned TextBuffers.
I do a check of not initialized arrays and init them.
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Pascal on September 19, 2016, 10:14:23 pm
By the way: If i disable "Outline (global)" i can not set "Outline" for the individual keyword as it is disabled.
Why is it disabled?
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on September 19, 2016, 11:28:54 pm
if outlining is disabled, then there is no point in being able to change its config.

It would not change anything.
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Pascal on September 20, 2016, 05:49:12 am
if outlining is disabled, then there is no point in being able to change its config.

It would not change anything.
Ah, i see. So you enable outlining with "Outline (global)" and then disable individual folds.
I thought it behaves more like enable/disable all folds.
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Pascal on September 23, 2016, 11:08:58 pm
Martin,

did you review the latest patch already?
 
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on September 23, 2016, 11:46:10 pm
not yet, been on your other patch
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on September 24, 2016, 01:50:10 pm
I get a range check, when I change Editor opts in the IDE.

Code: Pascal  [Select][+][-]
  1. #0 HANDLEERRORADDRFRAME(201, 0xc218e2, 0x102af864) at ..\inc\system.inc:1104
  2. #1 HANDLEERRORADDRFRAMEIND(201, 0xc218e2, 0x102af864) at ..\inc\system.inc:1123
  3. #2 fpc_dynarray_rangecheck(0x0, 0) at ..\inc\dynarr.inc:41
  4. #3 DOTEXTCHANGED(0x13e48e28, 1, 2589, 0) at syneditmarkupfoldcoloring.pas:737
  5. #4 TEXTCHANGED(0x13e48e28, 1, 2590, 0) at syneditmarkup.pp:442
  6. #5 TEXTCHANGED(0x140e6208, 1, 2590, 0) at syneditmarkup.pp:614
  7. #6 HIGHLIGHTERATTRCHANGED(0x1413e8a0, 0x13934108) at synedit.pp:8123
  8. #7 CALLNOTIFYEVENTS(0x139f45f8, 0x13934108) at lazmethodlist.pas:307
  9. #8 DEFHIGHLIGHTCHANGE(0x13934108, 0x13934108) at synedithighlighter.pp:1453
  10. #9 ENDUPDATE(0x13934108) at synedithighlighter.pp:1297
  11. #10 UPDATEHIGHLIGHTERS(0x118665d8, true) at mainbase.pas:1939
  12. #11 DOEDITOROPTIONSAFTERWRITE(0x118665d8, 0x1683d0, false) at main.pp:4863
  13.  

Code: Pascal  [Select][+][-]
  1.   // invalidate cache
  2.   for i := ToIdx(StartLine) to ToIdx(EndLine) do begin
  3.     FFirstCharacterColumn[i] := 0;
  4.     FEndLine[i] := 0;
  5.   end;
  6.  
FFirstCharacterColumn is empty (length - 0)

Startline = 1 and endline = linecount.

--------------------
Btw there is lots of commented out code. Still needed?


--------------------
Quote
Fixed! Added new patch which also uses NestList.NodeEndLine.
There are still various FHighlighter.FoldEndLine ?

------------------
And then there seems to be a bug outside your code. The paint code hits an assert, when I type Japanese Hiragana (double width chars)

Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on September 24, 2016, 03:17:21 pm
Code: Pascal  [Select][+][-]
  1. procedure TSynEditMarkupFoldColors.InitArrays;
  2. ....
  3.   Lines.AddChangeHandler(senrLineCount, @LinesChanged);
  4.   Lines.AddChangeHandler(senrHighlightChanged, @HighlightChanged);
  5.  

This is called from TSynEditMarkupFoldColors.SetLines (where it is matched by remove calls.)
But also from other locations. (Afaik duplicates are ignored, but still....)

Also
Code: Pascal  [Select][+][-]
  1. constructor TSynEditMarkupFoldColors.Create(ASynEdit: TSynEditBase);
  2. begin
  3.   inherited Create(ASynEdit);
  4.  
  5.   if Assigned(Lines) then begin
  6.     Lines.AddChangeHandler(senrLineCount, @LinesChanged);
  7. ....
  8.  
It can never be assigned.

This code should only be in SetLines and Destroy (a remove call is ignored, if the handler wasn't added.
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on September 24, 2016, 03:24:41 pm
InitArrays is also the wrong place for
Code: Pascal  [Select][+][-]
  1.   if Assigned(FHighlighter) then begin
  2.     FNestList := TLazSynEditNestedFoldsList.Create(Lines, FHighlighter);
  3.  

That code belongs ONLY into TSynEditMarkupFoldColors.HighlightChanged

There is a copy in Create, but I wonder, can create just call HighlightChanged?

--------------
Why does SetLines do FreeAndNil(FNestList); ?

FNestList.Lines needs to be updated in SetLines
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on September 24, 2016, 03:46:09 pm
You are using logical positions http://wiki.lazarus.freepascal.org/SynEdit#Logical.2FPhysical_caret_position

That causes the tab misalign.....

You should use Phys x pos.

You need to convert the first none space, into phys, and then on each line use that.
(for keywords (frame) you can use either log, or pos. Since they are for one line only, so the pos can not shift)

(It will not fix all problems, but it is a start)
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on September 24, 2016, 04:11:03 pm
Code: Pascal  [Select][+][-]
  1.  procedure AddVerticalLine(
  2. ....
  3.     z := Length(FFoldColorInfos);
  4.     SetLength(FFoldColorInfos, z+1);
  5.  
Better to set the lenght to the full needed lengh at start.

That should be "NestCount := FNestList.Count;"?

Well I guess up to NestCount. It can be less.
But then set it to maximum, and reduce at the end.

Or better yet, keep it at maximum (and also keep for next line) and store the real used len in a separate variable.

Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Pascal on September 25, 2016, 08:17:30 am
Code: Pascal  [Select][+][-]
  1. procedure TSynEditMarkupFoldColors.InitArrays;
  2. ....
  3.   Lines.AddChangeHandler(senrLineCount, @LinesChanged);
  4.   Lines.AddChangeHandler(senrHighlightChanged, @HighlightChanged);
  5.  

This is called from TSynEditMarkupFoldColors.SetLines (where it is matched by remove calls.)
But also from other locations. (Afaik duplicates are ignored, but still....)

InitArray is only called once It's the fix for cloned Buffers as SetLines isn't called for the clone SynEdit


Also
Code: Pascal  [Select][+][-]
  1. constructor TSynEditMarkupFoldColors.Create(ASynEdit: TSynEditBase);
  2. begin
  3.   inherited Create(ASynEdit);
  4.  
  5.   if Assigned(Lines) then begin
  6.     Lines.AddChangeHandler(senrLineCount, @LinesChanged);
  7. ....
  8.  
It can never be assigned.

This code should only be in SetLines and Destroy (a remove call is ignored, if the handler wasn't added.

Okay. So lines is always nil when constructing. I will remove this. This code was there before i began.
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Pascal on September 25, 2016, 08:21:26 am
InitArrays is also the wrong place for
Code: Pascal  [Select][+][-]
  1.   if Assigned(FHighlighter) then begin
  2.     FNestList := TLazSynEditNestedFoldsList.Create(Lines, FHighlighter);
  3.  

That code belongs ONLY into TSynEditMarkupFoldColors.HighlightChanged

There is a copy in Create, but I wonder, can create just call HighlightChanged?

--------------
Why does SetLines do FreeAndNil(FNestList); ?

FNestList.Lines needs to be updated in SetLines

It is there because TLazSynEditNestedFoldsList.Create(Lines, FHighlighter) depends on both Lines and Highlighter!
It has to be called when one of them changes.
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Pascal on September 25, 2016, 08:23:37 am
You are using logical positions http://wiki.lazarus.freepascal.org/SynEdit#Logical.2FPhysical_caret_position

That causes the tab misalign.....

You should use Phys x pos.

You need to convert the first none space, into phys, and then on each line use that.
(for keywords (frame) you can use either log, or pos. Since they are for one line only, so the pos can not shift)

(It will not fix all problems, but it is a start)

Okay. I never used tabs. I wil fix this.
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Pascal on September 25, 2016, 08:33:50 am
Code: Pascal  [Select][+][-]
  1.  procedure AddVerticalLine(
  2. ....
  3.     z := Length(FFoldColorInfos);
  4.     SetLength(FFoldColorInfos, z+1);
  5.  
Better to set the lenght to the full needed lengh at start.

That should be "NestCount := FNestList.Count;"?

Well I guess up to NestCount. It can be less.
But then set it to maximum, and reduce at the end.

Or better yet, keep it at maximum (and also keep for next line) and store the real used len in a separate variable.

I also thought about this as the array gets copied every time. I will fix this.
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on September 25, 2016, 02:14:54 pm
Quote
InitArray is only called once It's the fix for cloned Buffers as SetLines isn't called for the clone SynEdit
When the buffer was replaced from a shared editor, then there is no need for AddChangeHandler (handlers are copied to the new buffer)

Quote
It is there because TLazSynEditNestedFoldsList.Create(Lines, FHighlighter) depends on both Lines and Highlighter!
It has to be called when one of them changes.

You may have to do an FNestList.Clear
And FNestList.Lines := Lines // Despite lines do not change in todays SynEdit.


And in TSynEditMarkupFoldColors.HighlightChanged
  FNestList.Highlighter := FHighlighter;


Maybe rename it to TextBufferChanged?


Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Pascal on September 25, 2016, 02:27:50 pm
Quote
InitArray is only called once It's the fix for cloned Buffers as SetLines isn't called for the clone SynEdit
When the buffer was replaced from a shared editor, then there is no need for AddChangeHandler (handlers are copied to the new buffer)

So i only have to init the arrays, right? Is there a way to store them with the lines, so that the clone can also use it?
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on September 25, 2016, 02:44:32 pm
So i only have to init the arrays, right? Is there a way to store them with the lines, so that the clone can also use it?

And maybe clear the nestlist, probably not needed, but wont hurt.

Look at unit SynEditTextTabExpander class TSynEditStringTabData

No it is not automatically copied to the new textbuffer (the whole shared buffer needs some improvments.
But it will automatically grow/shrink if lines are inserted / deleted.

Not sure if it is a big advantage right now. Maybe leave it.
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Pascal on September 29, 2016, 02:15:23 am
Added new patch. Fixed most but not all (Recreation of FNestList is still done on Highlighter change).

Which editor properties did you change to get the range check error? I couldn' reproduce it.
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Pascal on October 05, 2016, 09:34:24 pm
I fixed the last points of your test (http://bugs.freepascal.org/file_download.php?file_id=25124&type=bug (http://bugs.freepascal.org/file_download.php?file_id=25124&type=bug)) and i found a possible problem:

When i change the highlighter it does a ScanRange and notifies all Markups of changed text before they even know about the new highlighter!

Code: Pascal  [Select][+][-]
  1. procedure TCustomSynEdit.SetHighlighter(const Value: TSynCustomHighlighter);
  2. begin
  3.     ...
  4.     fHighlighter := Value;
  5.     IncPaintLock;
  6.     try
  7.       // Ensure to free all copies in SynEit.Notification too
  8.       fMarkupHighCaret.Highlighter := Value;
  9.       fMarkupWordGroup.Highlighter := Value;
  10.       FFoldedLinesView.Highlighter := Value;
  11.       FPaintArea.Highlighter := Value;
  12.       FWordBreaker.Reset;
  13.       if fHighlighter<>nil then begin
  14.         fTSearch.IdentChars := fHighlighter.IdentChars;
  15.         FWordBreaker.IdentChars     := fHighlighter.IdentChars;
  16.         FWordBreaker.WordBreakChars := fHighlighter.WordBreakChars;
  17.       end else begin
  18.         fTSearch.ResetIdentChars;
  19.       end;
  20.       RecalcCharExtent;
  21.       ScanRanges; // Todo: Skip if paintlocked <------------------- ScanRanges calls TextChanged for all Markups (see below)
  22.       // There may not have been a scan
  23.       if fHighlighter <> nil then
  24.         FHighlighter.CurrentLines := FLines;
  25.       FLines.SendNotification(senrHighlightChanged, FLines, -1, -1); <--------------- notify about new highlighter
  26.     finally
  27.       DecPaintLock;
  28.     end;
  29.   end;
  30. end;
  31.  

Code: Pascal  [Select][+][-]
  1. procedure TCustomSynEdit.ScanRanges(ATextChanged: Boolean = True);
  2. begin
  3.   ...
  4.   // Todo: text may not have changed
  5.   if ATextChanged then
  6.     fMarkupManager.TextChanged(FChangedLinesStart, FChangedLinesEnd, FChangedLinesDiff);  <------- notify Markups of changed Text (FChangedLinesStart is 0 here)
  7.   TopView := TopView;
  8. end;

So maybe the notify should be done before ScanRange !? Or call ScanRange(false).

I could handle this situation as StartLine is 0 in DoTextChanged virtual procedure (as it should be 1 based). So i exit DoTextChanged
when i get a StartLine of 0.
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: x2nie on October 06, 2016, 12:45:22 pm
...

Code: Pascal  [Select][+][-]
  1. procedure TCustomSynEdit.SetHighlighter(const Value: TSynCustomHighlighter);
  2. begin
  3. ...
  4.       ScanRanges; // Todo: Skip if paintlocked <------------------- ScanRanges calls TextChanged for all Markups (see below)
  5. end;
  6.  

Code: Pascal  [Select][+][-]
  1. procedure TCustomSynEdit.ScanRanges(ATextChanged: Boolean = True);
  2. begin
  3.   if ATextChanged then
  4.     fMarkupManager.TextChanged(FChangedLinesStart, FChangedLinesEnd, FChangedLinesDiff);  <------- notify Markups of changed end;

So maybe the notify should be done before ScanRange !? Or call ScanRange(false).
I think the ScanRange(False);         will solve the potentially a problem.
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on October 07, 2016, 12:59:00 am
Yes it seem ScanRanges(False) should be correct. I cant think of any current markup that would break....

Quote
So maybe the notify should be done before ScanRange !? Or call ScanRange(false).
Looking at the code....

It seems that
Code: Pascal  [Select][+][-]
  1. FLines.SendNotification(senrHighlightChanged, FLines, -1, -1);
may never have been meant as "the highlighter was replaced"

It may go with the comment
Code: Pascal  [Select][+][-]
  1. // There may not have been a scan
meaning, it simply simulates the event for a full scan of the HL (in case it is needed)
In that case the order is correct, or does at least no matter.

Something else:
Code: Pascal  [Select][+][-]
  1.       fMarkupHighCaret.Highlighter := Value;
  2.       fMarkupWordGroup.Highlighter := Value;
  3.  
SynEdit shouldnt need to know individual markups.

One way would be to introduce senrHighlighterClassChanged.
But easier will be to put HighlighterChanged() the base class, and call MarkupManager.HighlighterChanged, that will call all Markups.
Any Markup in need of a HL, can react (the 2 existing Markup can retrieve the HL)
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Pascal on October 07, 2016, 05:57:10 am
It seems that
Code: Pascal  [Select][+][-]
  1. FLines.SendNotification(senrHighlightChanged, FLines, -1, -1);
may never have been meant as "the highlighter was replaced"

But -1, -1 is the only way to identify a highlighter change as senrHighlightChanged is also called many times for other reasons.
 
Something else:
Code: Pascal  [Select][+][-]
  1.       fMarkupHighCaret.Highlighter := Value;
  2.       fMarkupWordGroup.Highlighter := Value;
SynEdit shouldnt need to know individual markups.

One way would be to introduce senrHighlighterClassChanged.
But easier will be to put HighlighterChanged() the base class, and call MarkupManager.HighlighterChanged, that will call all Markups.
Any Markup in need of a HL, can react (the 2 existing Markup can retrieve the HL)

You can also modify those markups so that they register a notification handler for senrHighlightChanged (like TSynEditMarkupFoldColors does).
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on October 07, 2016, 04:16:01 pm
Quote
But -1, -1 is the only way to identify a highlighter change as senrHighlightChanged is also called many times for other reasons.
Is called to indicate for which lines the highlighting has changed.
-1,-1 should probably be "all", not sure why it was done that way.

Thats why change of HL object, should have its own notify. For now, it may be easy to solve for markup only. (by a method on markup)

Scan ranges, then notifes, once the HL has scanned. (before that, the node info on the HL is incorrect)

I will look at it...

----------------------
The original idea of TextChanged, was that a markup would simply sot a flag (store the min/max changed line); and only when need for painting it will calculate.

Non sure, but textChanged may be called for lines not even visible. Or (if triggered by user app, and without BeginUpdate):
TextChange may be called for the lines that are visible, but then before they ever get painted, the editor could scroll.
In that case Any work done by TextChange would just be a waste of cpu time, since the result in never painted.

Code: Pascal  [Select][+][-]
  1. SynEdit.TextBetweenPoint[Point(1, SynEdit.TopLine),Point(1, SynEdit.TopLine)] := 'foo';
  2. SynEdit.TopLine := SynEdit.TopLine + 200

But dont worry, if all else is ok, I will apply it, even with actions in TextChanged.
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on October 26, 2016, 03:55:05 am
long time, sorry ....

Still reviewing, but the patches are now certainly better than the existing code, so I will commit it (patch 15 is committed to svn), even if it needs more work (review pending).

---------------------------

FNestList2 isn't needed. And used incorectly.
In AddVerticalLine
Code: Pascal  [Select][+][-]
  1.       FNestList2.Line := ANode.LineIndex;
  2.       l := ToPos(FNestList2.NodeEndLine[0]);
  3.  
returns the endline of the most outer node (usually the unit...end / so end of file)

You were looking for
Code: Pascal  [Select][+][-]
  1.       l := ToPos(FNestList2.NodeEndLine[FNestList2.Count-1]);
Though that depends what node you want, if more than one opens on that line.

Or better (ANodeIdx is the index of the ANode, from the nestlist):
Code: Pascal  [Select][+][-]
  1.       l := ToPos(FNestList.NodeEndLine[ANodeIdx]);
  2.  

I did make those changes, before I commited

------------------------
You may want to check in DoTextChanged (line 731)
Code: Pascal  [Select][+][-]
  1.     if sfaOutline in FoldAction then begin
  2.       FNestList.Line := LineIndex;
  3.       l := ToPos(FNestList.NodeEndLine[FNestList.Count]);
  4.  
that is the most inner node before-or-on the current line, if any. (or crash if none, but I believe the if is only true if there are nodes.

You really should have the index at which the node was found. But then at the time FNestList may have changed....

----------------------
Why is there
SetCacheCount(100);
in create?

----------------------
your detection for shared buffer did no longer work, since the array always has some length (well it would have the length from the previous buffer anyway?)

I fixed that.
----------------------
need to check, that it does nothing, if not enabled (markup.RealEnabled)

also you probably need override RealEnabled, since you got more colors.... (the var should be FColors, not Colors). Not sure what (if any) is stored in MarkupInfo in your case.

Also in RealEnabled check if the Highlighter has support.....

----------------------
I started a test case.
But its only a dummy, to show how it can be done.
Also add only behaviour that is save not to change...
----------------------
more later......
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: guest58172 on October 27, 2016, 01:27:43 am
Two questions:
I've put a few captures of what looks a bit odd, even if I understand that's this new markup is WIP.
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Pascal on October 27, 2016, 12:44:17 pm
Which Highlighter did you use?
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: guest58172 on October 28, 2016, 06:36:56 am
Which Highlighter did you use?

This one: https://github.com/BBasile/Coedit/blob/master/src/ce_d2syn.pas

Explanations for the screenshots:

(1): https://github.com/BBasile/Coedit/blob/master/src/ce_d2syn.pas#L503
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Pascal on October 28, 2016, 08:03:05 pm
Later on there will be an extension to the configuration of fold coloring to disable individual nodes from coloring (your comments):
Martin_fr posted a patch on mantis http://bugs.freepascal.org/file_download.php?file_id=24855&type=bug (http://bugs.freepascal.org/file_download.php?file_id=24855&type=bug) with
this extensions.
In your Highlighter you can decide which nodes should be colored and set options:
See TSynPasSyn.DoInitNode in unit SynHighlighterPas and sfaOutline...-FoldActions.
Remove sfaOutline from aActions to disable coloring.
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Pascal on December 02, 2016, 12:44:46 pm
Martin,

there is an other bug remainig: array FFoldColorInfos in SynEditMarkupFoldColoring is too small/not dynamic
I put a patch on Mantis: http://bugs.freepascal.org/view.php?id=31049 (http://bugs.freepascal.org/view.php?id=31049)
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Pascal on January 09, 2017, 04:09:26 pm
Martin,

could you please commit this patch?
there is an other bug remainig: array FFoldColorInfos in SynEditMarkupFoldColoring is too small/not dynamic
I put a patch on Mantis: http://bugs.freepascal.org/view.php?id=31049 (http://bugs.freepascal.org/view.php?id=31049)

I would like to continue developement to bring the vertical lines to the else-part too.
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on January 24, 2017, 03:01:51 am
Something I noted:
Markups are supposed to do NO work, if they are not enabled (or if there colors are ALL off)

For most markups that is covered by the manager, because the GetNext.... functions are not called.

But TextChanged is still called. Many markup set only a flag in there "FFirstLineChanged:=.... FlastLine.....".
That is ok (not much work)

But yours does a lot of work in
Code: Pascal  [Select][+][-]
  1.      procedure TextBufferChanged(Sender: TSynEditStrings; aIndex, aCount: Integer
  2.     procedure DoTextChanged({%H-}StartLine, EndLine, {%H-}ACountDiff: Integer); override; // 1 based
  3.     procedure SetLines(const AValue: TSynEditStrings); override;
  4.     procedure LinesChanged(Sender: TSynEditStrings; aIndex, aCount: Integer);
  5.     procedure HighlightChanged(Sender: TSynEditStrings; aIndex, aCount: Integer);
  6.  

So they should start with
Code: Pascal  [Select][+][-]
  1.   if not ( Enabled and MarkupInfo.IsEnabled) then exit;

Of course then you need to listen to
  DoEnabledChanged
  MarkupChanged
and if you get enabled, then initialize.

Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Pascal on January 24, 2017, 06:06:15 am
Something I noted:
Markups are supposed to do NO work, if they are not enabled (or if there colors are ALL off)

For most markups that is covered by the manager, because the GetNext.... functions are not called.

But TextChanged is still called. Many markup set only a flag in there "FFirstLineChanged:=.... FlastLine.....".
That is ok (not much work)

But yours does a lot of work in
Code: Pascal  [Select][+][-]
  1.      procedure TextBufferChanged(Sender: TSynEditStrings; aIndex, aCount: Integer
  2.     procedure DoTextChanged({%H-}StartLine, EndLine, {%H-}ACountDiff: Integer); override; // 1 based
  3.     procedure SetLines(const AValue: TSynEditStrings); override;
  4.     procedure LinesChanged(Sender: TSynEditStrings; aIndex, aCount: Integer);
  5.     procedure HighlightChanged(Sender: TSynEditStrings; aIndex, aCount: Integer);
  6.  

So they should start with
Code: Pascal  [Select][+][-]
  1.   if not ( Enabled and MarkupInfo.IsEnabled) then exit;

Of course then you need to listen to
  DoEnabledChanged
  MarkupChanged
and if you get enabled, then initialize.

Okay, i will put this in.
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Pascal on January 24, 2017, 06:11:24 am
I found an other bug and i have no clue what went wrong in my code.
I attached a sample prog. Start it, place the cursor after {$ifdef DEBUG} and press ENTER.
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on January 24, 2017, 03:05:21 pm
I found an other bug and i have no clue what went wrong in my code.
I attached a sample prog. Start it, place the cursor after {$ifdef DEBUG} and press ENTER.
I have a look in a bit...

Maybe you did not call "clear" on the nested fold list.

If possible this should be added to the testcase first. So the test will fail now, and the test shows that it is fixed.
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Pascal on January 24, 2017, 07:21:00 pm
Maybe you did not call "clear" on the nested fold list.

Code: Pascal  [Select][+][-]
  1.   y := ToIdx(aRow);
  2.   FNestList.Clear;
  3.   FNestList.Line := y;
  4.  

Calling Clear before setting Line did it. I thought setting Line would also clear the NestList.
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on January 24, 2017, 07:42:06 pm
Calling Clear before setting Line did it. I thought setting Line would also clear the NestList.

the list re-uses info.
If you are at line n, and it scans back 100 lines to find outer nodes => then if you set line n+1 it can still use the info from the 100 scanned lines.
So that reduces a lot of work.

But if the HL changed (if text changed, and HL scanned) then you need to clear.

You should clear, ONCE, at the first line that is painted. Following lines should only set the number (or it can get a lot slower).

Maybe best to introduce "BeginMarkup" same as "FinishMArkup" but called before the very first line.
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Pascal on January 25, 2017, 06:09:03 am
You should clear, ONCE, at the first line that is painted. Following lines should only set the number (or it can get a lot slower).

Maybe best to introduce "BeginMarkup" same as "FinishMArkup" but called before the very first line.

I already had it in EndMarkup:
Code: Pascal  [Select][+][-]
  1. procedure TSynEditMarkupFoldColors.EndMarkup;
  2. begin
  3.   {$IFDEF SynEditMarkupFoldColoringDebug}
  4.   //DebugLn('EndMarkup');
  5.   {$ENDIF}
  6.   inherited EndMarkup;
  7.   if not Assigned(FHighlighter) then exit;
  8.   FNestList.Clear; // for next markup start
  9. end;
  10.  

I also tried it in the HighlighChanged handler: no luck.

The place where it seems to work is DoTextChanged. Could this be okay to clear it there?

Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on January 25, 2017, 11:12:02 pm
Well EndMarkup would work (ignoring lines_changed and other callbacks)

Lets look at the reasons.
- If the nestedList has a line set, and then the highlighter ScansRanges, then the list must be cleared.

The highlighter scans, between EndMarkup and the next GetNextTokenPos... (in paint).
But if the list was cleared before the scan, then it did not have a line, and it is fine.

-----------------
Now if other code also set a line to the list, then you must check, if the HL can scan after that code before the next use of the list.

--------------
calling clear in TextChanged may work, because it (should) always be called, after the HL scanned.)

For safety a clear in (a not yet existent) StartMarkup, would be good.

-------------
Also check if you use it in any of the other hooks you register, and if they need to clear it.

------------
And last check that you ONLY use it, when there is no hl scan pending (because it is not save if the lines where changed, but the HL did not yet scan)

Unfortunatel I just saw, that SYnEdit calls TextChanged in  TCustomSynEdit.HighlighterAttrChanged, and I am not sure that this is save.

Code: Pascal  [Select][+][-]
  1.   HL.CurrentLines:= Synedit.lines // or similar
  2.   res := HL.NeedScan,
  3.  

If a scan is pending exit TextChanged. IT will be called again.

As for other calllbacks, they can not operate if they need the list and a scan is pending.

Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Pascal on January 26, 2017, 01:10:22 pm
Something I noted:
Markups are supposed to do NO work, if they are not enabled (or if there colors are ALL off)

For most markups that is covered by the manager, because the GetNext.... functions are not called.

But TextChanged is still called. Many markup set only a flag in there "FFirstLineChanged:=.... FlastLine.....".
That is ok (not much work)

But yours does a lot of work in
Code: Pascal  [Select][+][-]
  1.      procedure TextBufferChanged(Sender: TSynEditStrings; aIndex, aCount: Integer
  2.     procedure DoTextChanged({%H-}StartLine, EndLine, {%H-}ACountDiff: Integer); override; // 1 based
  3.     procedure SetLines(const AValue: TSynEditStrings); override;
  4.     procedure LinesChanged(Sender: TSynEditStrings; aIndex, aCount: Integer);
  5.     procedure HighlightChanged(Sender: TSynEditStrings; aIndex, aCount: Integer);
  6.  

So they should start with
Code: Pascal  [Select][+][-]
  1.   if not ( Enabled and MarkupInfo.IsEnabled) then exit;

Of course then you need to listen to
  DoEnabledChanged
  MarkupChanged
and if you get enabled, then initialize.

Strange! DoMarkupChanged toggles very often between enabled and disabled.
Why is the MarkupInfo of the OutlineMarkup changed by the highlighter?

Code: Pascal  [Select][+][-]
  1. #0 TSYNEDITMARKUPFOLDCOLORS__CHECKENABLED(<error reading variable>) at C:\freepascal\laz\components\synedit\syneditmarkupfoldcoloring.pas:683
  2. #1 TSYNEDITMARKUPFOLDCOLORS__MYMARKUPINFOCHANGED(0x36af890, <error reading variable>) at C:\freepascal\laz\components\synedit\syneditmarkupfoldcoloring.pas:956
  3. #2 TSYNHIGHLIGHTERATTRIBUTES__DOCHANGE(<error reading variable>) at C:\freepascal\laz\components\synedit\synedithighlighter.pp:1241
  4. #3 TLAZSYNCUSTOMTEXTATTRIBUTES__CHANGED(<error reading variable>) at C:\freepascal\laz\components\synedit\synedithighlighter.pp:789
  5. #4 TLAZSYNCUSTOMTEXTATTRIBUTES__SETFOREGROUND(536870911, <error reading variable>) at C:\freepascal\laz\components\synedit\synedithighlighter.pp:745
  6. #5 TSYNEDITMARKUPFOLDCOLORS__GETMARKUPATTRIBUTEATROWCOL(5, {PHYSICAL = 1, LOGICAL = 1, OFFSET = 0}, {ISRTL = false, PHYSLEFT = 1431655765, PHYSRIGHT = 1431655765, LOGFIRST = 1431655765, LOGLAST = 1431655765}, <error reading variable>) at C:\freepascal\laz\components\synedit\syneditmarkupfoldcoloring.pas:230
  7. #6 TSYNEDITMARKUP__MERGEMARKUPATTRIBUTEATROWCOL(5, {PHYSICAL = 1, LOGICAL = 1, OFFSET = 0}, {PHYSICAL = 5, LOGICAL = 5, OFFSET = 0}, {ISRTL = false, PHYSLEFT = 1431655765, PHYSRIGHT = 1431655765, LOGFIRST = 1431655765, LOGLAST = 1431655765}, 0x802f6e8, <error reading variable>) at C:\freepascal\laz\components\synedit\syneditmarkup.pp:435
  8. #7 TSYNEDITMARKUPMANAGER__MERGEMARKUPATTRIBUTEATROWCOL(5, {PHYSICAL = 1, LOGICAL = 1, OFFSET = 0}, {PHYSICAL = 5, LOGICAL = 5, OFFSET = 0}, {ISRTL = false, PHYSLEFT = 1431655765, PHYSRIGHT = 1431655765, LOGFIRST = 1431655765, LOGLAST = 1431655765}, 0x802f6e8, <error reading variable>) at C:\freepascal\laz\components\synedit\syneditmarkup.pp:579
  9. #8 TLAZSYNPAINTTOKENBREAKER__GETNEXTHIGHLIGHTERTOKENEX({TK = {TOKENSTART = 0x8022464 'uses', TOKENLENGTH = 4, TOKENATTR = 0x36d5958}, ATTR = 0x802f528, STARTPOS = {PHYSICAL = 1, LOGICAL = 1, OFFSET = 0}, ENDPOS = {PHYSICAL = 5, LOGICAL = 5, OFFSET = 0}, PHYSICALCHARSTART = 1, PHYSICALCLIPSTART = 1, PHYSICALCHAREND = 5, PHYSICALCLIPEND = 5, RTLINFO = {ISRTL = false, PHYSLEFT = 1431655765, PHYSRIGHT = 1431655765, LOGFIRST = 1431655765, LOGLAST = 1431655765}, RTLEXPANDEDEXTRABYTES = 1431655765, RTLHASTABS = 85, RTLHASDOUBLEWIDTH = 85, EXPANDEDEXTRABYTES = 0, HASTABS = false, HASDOUBLEWIDTH = false, NEXTPOS = {PHYSICAL = 5, LOGICAL = 5, OFFSET = 0}, NEXTRTLINFO = {ISRTL = false, PHYSLEFT = 0, PHYSRIGHT = 0, LOGFIRST = 1, LOGLAST = 1}}, <error reading variable>) at C:\freepascal\laz\components\synedit\lazsyntextarea.pp:417
  10. #9 PAINTLINES(0x358f658) at C:\freepascal\laz\components\synedit\lazsyntextarea.pp:1721
  11. ...
  12.  

----
Does this option disable the OutlineMarkup?
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Pascal on January 27, 2017, 08:48:18 pm
Strange! DoMarkupChanged toggles very often between enabled and disabled.
Why is the MarkupInfo of the OutlineMarkup changed by the highlighter?

Code: Pascal  [Select][+][-]
  1. #0 TSYNEDITMARKUPFOLDCOLORS__CHECKENABLED(<error reading variable>) at C:\freepascal\laz\components\synedit\syneditmarkupfoldcoloring.pas:683
  2. #1 TSYNEDITMARKUPFOLDCOLORS__MYMARKUPINFOCHANGED(0x36af890, <error reading variable>) at C:\freepascal\laz\components\synedit\syneditmarkupfoldcoloring.pas:956
  3. #2 TSYNHIGHLIGHTERATTRIBUTES__DOCHANGE(<error reading variable>) at C:\freepascal\laz\components\synedit\synedithighlighter.pp:1241
  4. #3 TLAZSYNCUSTOMTEXTATTRIBUTES__CHANGED(<error reading variable>) at C:\freepascal\laz\components\synedit\synedithighlighter.pp:789
  5. #4 TLAZSYNCUSTOMTEXTATTRIBUTES__SETFOREGROUND(536870911, <error reading variable>) at C:\freepascal\laz\components\synedit\synedithighlighter.pp:745
  6. #5 TSYNEDITMARKUPFOLDCOLORS__GETMARKUPATTRIBUTEATROWCOL(5, {PHYSICAL = 1, LOGICAL = 1, OFFSET = 0}, {ISRTL = false, PHYSLEFT = 1431655765, PHYSRIGHT = 1431655765, LOGFIRST = 1431655765, LOGLAST = 1431655765}, <error reading variable>) at C:\freepascal\laz\components\synedit\syneditmarkupfoldcoloring.pas:230
  7. #6 TSYNEDITMARKUP__MERGEMARKUPATTRIBUTEATROWCOL(5, {PHYSICAL = 1, LOGICAL = 1, OFFSET = 0}, {PHYSICAL = 5, LOGICAL = 5, OFFSET = 0}, {ISRTL = false, PHYSLEFT = 1431655765, PHYSRIGHT = 1431655765, LOGFIRST = 1431655765, LOGLAST = 1431655765}, 0x802f6e8, <error reading variable>) at C:\freepascal\laz\components\synedit\syneditmarkup.pp:435
  8. #7 TSYNEDITMARKUPMANAGER__MERGEMARKUPATTRIBUTEATROWCOL(5, {PHYSICAL = 1, LOGICAL = 1, OFFSET = 0}, {PHYSICAL = 5, LOGICAL = 5, OFFSET = 0}, {ISRTL = false, PHYSLEFT = 1431655765, PHYSRIGHT = 1431655765, LOGFIRST = 1431655765, LOGLAST = 1431655765}, 0x802f6e8, <error reading variable>) at C:\freepascal\laz\components\synedit\syneditmarkup.pp:579
  9. #8 TLAZSYNPAINTTOKENBREAKER__GETNEXTHIGHLIGHTERTOKENEX({TK = {TOKENSTART = 0x8022464 'uses', TOKENLENGTH = 4, TOKENATTR = 0x36d5958}, ATTR = 0x802f528, STARTPOS = {PHYSICAL = 1, LOGICAL = 1, OFFSET = 0}, ENDPOS = {PHYSICAL = 5, LOGICAL = 5, OFFSET = 0}, PHYSICALCHARSTART = 1, PHYSICALCLIPSTART = 1, PHYSICALCHAREND = 5, PHYSICALCLIPEND = 5, RTLINFO = {ISRTL = false, PHYSLEFT = 1431655765, PHYSRIGHT = 1431655765, LOGFIRST = 1431655765, LOGLAST = 1431655765}, RTLEXPANDEDEXTRABYTES = 1431655765, RTLHASTABS = 85, RTLHASDOUBLEWIDTH = 85, EXPANDEDEXTRABYTES = 0, HASTABS = false, HASDOUBLEWIDTH = false, NEXTPOS = {PHYSICAL = 5, LOGICAL = 5, OFFSET = 0}, NEXTRTLINFO = {ISRTL = false, PHYSLEFT = 0, PHYSRIGHT = 0, LOGFIRST = 1, LOGLAST = 1}}, <error reading variable>) at C:\freepascal\laz\components\synedit\lazsyntextarea.pp:417
  10. #9 PAINTLINES(0x358f658) at C:\freepascal\laz\components\synedit\lazsyntextarea.pp:1721
  11. ...
  12.  
Stupid me!
TSYNEDITMARKUPFOLDCOLORS.GETMARKUPATTRIBUTEATROWCOL: MarkupInfo is cleared when no outlinecoloring occurs!

MarkupInfo is uesed internaly to respond to TSYNEDITMARKUPFOLDCOLORS.GETMARKUPATTRIBUTEATROWCOL.
So using MarkupInfo.IsEnabled doesn't make any sense with this markup, right?
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on January 27, 2017, 10:46:51 pm
you either need to have your own private markup, or check if you can override the methods that checks.

But because markup manager always listens to your primary markup, probably create a private instance.

At some time you may want to publish several markups, so people can configure colors.
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on January 29, 2017, 05:32:53 pm
Calling Clear before setting Line did it. I thought setting Line would also clear the NestList.

I think I figured out what caused the issue. (not debugged, just assumption)

Now as I said NestList has a cache. And if anything is in the cache, and the HL does change/scan then the cache is wrong.

Therefore if the HL changes, then "clear" is needed.
"clear" can be before or after the HL changes.
- before: the cache is empty, it can not become wrong by the changes
- after: you get rid of the wrong cache.

---------------------
Putting "clear" in EndMarkup was for the lists use in GetNextMarkupColAfterRowCol (or other methods used during paint).

The cache is cleaned after the paint, then the HL can make changes, come to the next paint, the cache is still empty....

The strange thing is why it failed in TextChanged, because TextChanged gets called AFTER the HL changed.
So the flow then is
- paint: after this list will have cache
- EndMarkup: clear cache
- HL changes: but cache is empty
- TextChanged: all fine, cache was empty / cache will be filled after
- next paint: cache is full, but there was no HL change.

So what went wrong?

Well Paint event can be skipped, or there could be several edits (without BeginUpdate).
Then you get
- paint: after this list will have cache
- EndMarkup: clear cache
- HL changes: but cache is empty
- TextChanged: all fine, cache was empty / cache will be filled after
- HL changes: cache is FULL
- TextChanged: invalid cache !!!!

By clearing either after or before TextChanged, this will be fixed.
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Pascal on January 31, 2017, 09:18:41 pm
Grrr,

due to EndPascalCodeFoldBlockLastLine the vertical lines are not drawn in an empty line before "else".
Especially with "trim trailing spaces".

So what is the better solution now:
a) using EndPascalCodeFoldBlockLastLine and find a workaround in the outline markup or
b) use EndPascalCodeFoldBlock and find a way to distinguish explicit from implicit closing token in the outline markup to not color "otherwise" and "case-else" which have no sfaOutline?
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Pascal on February 01, 2017, 05:56:16 am
b) is nearly impossible as i do not get the opening cbftCaseElse in the node list in TLazSynEditNestedFoldsList

For the time beeing i fixed this by only uses EndPascalCodeFoldBlockLastLine before cfbtCaseElse
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Pascal on February 01, 2017, 08:38:33 am
found a solution for a):
ignoring close blocks at end of line, so the vertical line will still be drawn (as the block is no more closed before the column of the vertical line)

----
Now we have one issue left. Color should only be kept (sfaOutlineKeepLevel) if new block starts on the same line.
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on February 01, 2017, 11:11:46 am
Actually, only "then" and "else" should keep (never mind the line)

"if" should not keep, not even on the same line...

That may have to be fixed in the HL, by splitting cfbtIfThen into 2.

Just trying to see, if there is a nice way to keep the old style depending on markup enabled or not. (only one blocktype uses less memory)

-----------------
Will revert in a bit.
 
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Pascal on February 01, 2017, 11:19:45 am
"if" should not keep, not even on the same line...
But here "else if" should keep the same color (if starts on same column):
This is only possible when "if" keeps color on the same line.
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Pascal on February 01, 2017, 11:26:20 am
I thought of extending FoldActions with
sfaOutlineKeepLevelOnSameLine (for cfbtIfThen,cfbtForDo,cfbtWhileDo,cfbtWithDo,cfbtIfElse) and
sfaOutlineMergeLevelOnWrongCol (for cfbtIfElse)
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Pascal on February 01, 2017, 11:32:21 am
"if" should not keep, not even on the same line...
This looks ugly  %)
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on February 01, 2017, 12:28:20 pm
what happens when
Code: Pascal  [Select][+][-]
  1. if foo then
  2.   if bar then x
  3.   else y
  4.  

"if" is on a new line, but it is the *same" statement, as "else if" on one line. Should it not have the same color?

--------------------------
The problem here, is to figure out what is hl, and what is markup.

1) The HL should provide the *minimum* information necessary.
2) The markup should not know about pascal (as the feature is language agnosting)


The markup can however be configure-able for different languages. You can have additional config, that can be matched against each cfbt* type.

Then you can also decide, when/when not a "begin" should/should not keep color.

---------------
By the way I would not say the last example looks ugly.

The diff color is helpful. You can immediately see which "if" the "else" belongs too.
If it where all yellow, then it would be unclear.
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Pascal on February 01, 2017, 12:34:30 pm
what happens when
Code: Pascal  [Select][+][-]
  1. if foo then
  2.   if bar then x
  3.   else y
  4.  

"if" is on a new line, but it is the *same" statement, as "else if" on one line. Should it not have the same color?
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Pascal on February 01, 2017, 12:37:52 pm
By the way I would not say the last example looks ugly.

The diff color is helpful. You can immediately see which "if" the "else" belongs too.
If it where all yellow, then it would be unclear.

You are right, you get more information! This should be configurable, so the user can decide.
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on February 01, 2017, 12:52:45 pm
You will be amazed how many different people will want many different colors....


For example, I would like colors by keyword.

That is
- try except: always red, nested in a different shade of red (1 or 2 shades, then repeat)
- "if" always green (nested with shades of green)
- "else" yellow
- loops: blue

Also I would have the keyword underlined in color, but not change the font color. Or maybe have they keyword not changed at all.

Also maybe only outline if/while , if they have a begin / or if they have nested if/while ....

There are millions of ways.....
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Pascal on February 01, 2017, 01:41:20 pm
I managed to get it look an behave like CNPack (at least from what i've seen on the screenshots) (Patch V20)
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Pascal on February 01, 2017, 01:59:33 pm
You will be amazed how many different people will want many different colors....


For example, I would like colors by keyword.

That is
- try except: always red, nested in a different shade of red (1 or 2 shades, then repeat)
- "if" always green (nested with shades of green)
- "else" yellow
- loops: blue

Also I would have the keyword underlined in color, but not change the font color. Or maybe have they keyword not changed at all.

Also maybe only outline if/while , if they have a begin / or if they have nested if/while ....

There are millions of ways.....

Next step is to publish MarkupInfo for each color. So you can decide the colors (frame, text, background), underline, style.
If you whant to have colors by keyord we need multiple arrays of MarkuInfo and we have to store the desired color array in the FoldAction
and we also have to have levels per color array.
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on February 01, 2017, 03:23:09 pm
Quote
Next step is to publish MarkupInfo for each color. So you can decide the colors (frame, text, background), underline, style.
If you whant to have colors by keyord we need multiple arrays of MarkuInfo and we have to store the desired color array in the FoldAction
and we also have to have levels per color array.

Actually no it does not go into FoldConfig....

Markup should not know about the language, yes, but... that refers to hard coded.

There be nothing wrong with having a config like
foo.ApplyToBlockType: integer;

and in IDE/User code setting
foo.ApplyToBlockType:= ord(cfbtIfThen);

This may be a list of integers, since there probably be groups.
Or you have the group mapping separate. (map each block-type to a group)
---------------------------

Then either have multiple lists of markup info, or subclass Markupinfo and add a property (either  ApplyToBlockType or ApplyToBlockGROUP (if you have group mapping)

You can then internally filter the markup list into a list for each group.

------------------------------
The other challenge is to find a good representation in IDE options. So users can easily set it up.

I once had the problem with mouse-options. I ended up with 2 pages. One for easy config. And an advanced one....



Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Pascal on February 01, 2017, 03:30:43 pm
Do you think we can fix the state of the markup like it is now (patch v20, like in the pictures)?
Then i can go on with configuring colors (stage 1).
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on February 01, 2017, 04:17:01 pm
havent looked at the patch yet. asap
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Pascal on April 21, 2017, 08:34:27 pm
Martin,

how can we add the posibility for the user to change the colors?

I could think of having an array of 10 TSynSelectedColor (MarkupInfo1..10 or an array).
If the MarkupInfo is enabled (IsEnabled) it will be used for outline coloring.

Where and how can i add the functionality to the preferences?
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on April 21, 2017, 09:21:56 pm
Fixed amount of colors is probably easiest. Before looking at how, I suggest to plan what. And that divides into what now, and what maybe later.

There may be plenty of requests what others want, and if some of them will be considered then better to design it flexible.

Ideas that I would have:

1) make the number of colors (before rotate) configurable (collection?)
At least on the markup class, the IDE option dialog may show fixed amount for now, since that will be considerable easier.

2) 2 markup info per color.
- One for the keyword
- One for the horiz line
e.g. I would like the keyword in my source to either have no highlight at all, or just a bottom-border (underline)
So you need an extra color for the vertical line.

3) Not sure, might go to far, but different color-sets for different block types. eg red-ish reserved for try except.

Depending on that you can decide what to add to the module.

IDE config is a bit more complex.
Search the svn history for when I added a color to ide/EditorOptions

All colors are stored on the highlighter
In the IDE that is in ide/SourceMarks TAdditionalHilightAttribute. Which is an enum. and that is why it will be easier to have a fixed amount in the IDE (yet there could be a config to only use n out of the overall available).

TAdditionalHilightAttribute is then used a lot in ide/EditorOptions to define names and other info

And at last have a look at TColorSchemeLanguage.ApplyTo
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on April 21, 2017, 09:26:28 pm
Or you create a separate option page/tab. Like "user defined markup"
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: mtournay on February 11, 2021, 06:20:39 pm
Hi there

Can someone, plz, lpz PLEASE create a 'tip of the day' on front lazarus-ide page so people can find the checkbox to tuen on this amazing functionnality ??

Every time I had to reinstall Lazarus on a new computer, I lost tons of minutes finding this thread, and then finding screen capture inside the thread....

Tanks a lot

Michel
Title: Re: Invite to Colorizing TSynEdit ! {Improving Editor for Editing Complex Codes}
Post by: Martin_fr on February 11, 2021, 06:51:07 pm
Can someone, plz, lpz PLEASE create a 'tip of the day' on front lazarus-ide page so people can find the checkbox to tuen on this amazing functionnality ??

"fold outline" or "user defined markup"?

"user defined markup" already has a T.o.D.


https://svn.freepascal.org/svn/lazaruswebsite/trunk
Tips are in the file tips.txt
Images can be between 300*200 and 400*200 / or will be scaled by the browser

Feel free to submit an addition.

Links to the wiki, not the forum.
e.g. https://wiki.lazarus.freepascal.org/IDE_Window:_Editor_User_Defined_Words


Btw, while (unfortunately) not all config files can be copied, the file EditorOptions.xml can be copied to the new install.
See menu: View > Ide Internals > about IDE => find primary config path => In that folder you find EditorOptions.xml

Patches for an export/import of config will be welcome.
TinyPortal © 2005-2018