Writing a TMemo widget from scratch

[Tommi]

If I compile TMemo with GTK2 widgets, the result isn't capable to manage more than 1 mb of text. If I compile it with QT it works a lot better with big files, anyway for license reason, I wouldn't use it.

So the idea would be to write it.
So I'd need to understand how a text editor works at low level. Does it create a big BMP showning just a slice of it ? So when I scroll it, the image is already in memory ?

Or does it design the canvas just with the text that I'm seeing right now and when I scroll it, the cpu redesign it instant by instant ?

And caret how is managed ?

Thank you

[TRon]

So I'd need to understand how a text editor works at low level. Does it create a big BMP showning just a slice of it ? So when I scroll it, the image is already in memory ?

Uhm where to begin ? Let me try keep it simple.

No, a text editor does not contain a bmp (sliced or otherwise) that contains all the text. A control has a canvas that has a visible part. At runtime the text will be rendered to that canvas depending on what part of the text is actually visible.

Now, having said that and generalizing a bit here, it might depend on the used underlying widgetset how things are rendered. Also depends on the control. A custom control renders the text itself while a widget control usually uses the system to render the text.

Or does it design the canvas just with the text that I'm seeing right now and when I scroll it, the cpu redesign it instant by instant ?

That is usually the way, yes.

And caret how is managed ?

Depends on the widgetset. It is an implementation detail.

A (plain) TMemo is not very capable of being used as an editor but it depends a bit on the needs for the editor. For simple notepad like experience it would be enough but if an editor requires things like linenumbers, gutters,  multicaret edits etc you are far better of using a control such as for example synedit (there exist more of such controls).

[paule32]

A starting Point could be the win32api.
Take a look to CreateWindow or CreateWindowEx.
Then take a look to the Classes of such Windows - in your case EDIT.
Then you can take a look to set the Font per HDC (Device Context).

Then you can consider to left the old fashion and jump to Class RICHEDIT.
RichEdit comes with a range of Version's

[n7800]

If I compile TMemo with GTK2 widgets, the result isn't capable to manage more than 1 mb of text. If I compile it with QT it works a lot better with big files, anyway for license reason, I wouldn't use it.

If this is the only problem, you can try the TSynEdit component - this is the component used by the IDE's text editor. It handles large files well and has a lot of features.

You can simply insert it into a form as TMemo, or try more complex customization methods. There are many example projects in the "C:\lazarus\examples\SynEdit\" folder.

But it only supports monospace fonts.

[Martin_fr]

A text editor will paint itself using something like TextOut (or any other similar API).

It will in most cases only have to repaint small bits of its window. So it must be able to paint any such sub-area. Though canvases have clipping, which means you can paint a bit more sometimes (e.g. you wouldn't try to paint half a char-glyph).

This is painted on the canvas, wich is normally persistent as long as it is visible on the screen. E.g. scrolling can often move the existing visible area on the screen, and fill in only the new text.

So far so good and really simple.

But even with monospaced fonts you have chars of different width. And must handle that. And you must handle text representation in unicode (even with utf16 or utf32) - assuming you will display more than just ASCII a..z, 0..9.
There are for example combining codepoints.

Tabs, you need to expand yourself...

And there is LTR and RTL text. That needs to be rendered properly. Well if you always output entire lines, and don't need line wrapping, then the OS likely will do that for you.

You need to at least have the concept of a text cursor and implement that.

And then likely moving that cursor, and modifying the text buffer.

All in all there will be a lot of tasks that you will need to master to write a text editor.

On the other hand, its a nice challenge, and you can learn a lot doing it. If you have time and commitment.


There are several open source text editors

TSynEdit
ATSynEdit (google)
likely some rich text editor, iirc.

[paule32]

old fashion is, to draw parts of the Text into a back-buffer that is allocated only for holding the memory data (text).
In this memory data all magic is done (records/classes of Objects for the Objects that shall be paint on the Canvas.

This back-buffer is the heart of all (update the Objects).

Then the back-buffer memory data will be used to render to the front-buffer (the Canvas) on the fly.

Many optimized steps will go into the back-buffer like:
- read from a Text-Buffer-Position (File data)
- read to a Text-Buffer-Position (File data, too)

The last thing is the Selection of the Objects (Text-Data) - which means, you have to handle mouseDown, mouseMove on the Canvas.

Not to forget the Update of the back-buffer data on each key-board press char.

[paule32]

to give you a insight on how effort is necassarry, to creare the window:
https://github.com/paule32/SWIPL_ZWAP/raw/main/img/term03.png

with this Code:
https://github.com/paule32/SWIPL_ZWAP/blob/fddf49436b8521744d6dde473f0256911f0df092/src/cc/prolog.cc#L5724

[Martin_fr]

to give you a insight on how effort is necassarry, to creare the window:
https://github.com/paule32/SWIPL_ZWAP/raw/main/img/term03.png

with this Code:
https://github.com/paule32/SWIPL_ZWAP/blob/fddf49436b8521744d6dde473f0256911f0df092/src/cc/prolog.cc#L5724

But here you could use the LCL. That will do all the API abstraction. You just need to create a descendant of  TCustomControl and implement Paint, and handling keys, input, textbuffer, cursor, undo, ....

[paule32]

@Martin
This shall be an Example, why the TS shall use existing stuff and NOT doing native and possible voodoo programming.
Nobody coding from slime brew up.

Therefore it give Frameworks.

I don't know, why the TS do this...

[Tommi]

I did some test with TSynEdit and it works good.
I did some test with this code:

Code: [Select]

procedure TForm1.Button1Click(Sender: TObject);
var
  a,b:integer;
  c:string='';
begin
  for a:= 1 to 1000000 do
  begin
    for b:=0 to 9 do
    begin
      c:=c+inttostr(b);
    end;
    c:=c+' ';
    //if a mod 10=0 then c:=c+#10; //Improves performance
  end;
  synedit1.Text:=c;
end;

procedure TForm1.FormCreate(Sender: TObject);
begin
    TLazSynEditLineWrapPlugin.Create(SynEdit1);
end;

I noticed that this row : if a mod 10=0 then c:=c+#10; allows to improve a lot the performances. I understand why: it becomes more easy to compute the position of characters, because the position of any character depends on the position of the last NewLine. So if the NewLine is near ,TSynEdit must iterate less times to elaborate the right position of visualized characters.

So, because sometime I have to open big SVG with base64 data inside, it think to insert random newlines into the big files, so I can edit data and when I save the fake Newlines are removed.

Is it a good idea ? Or may be there is some function similar already inside the widget ?

[paule32]

you would like to work on SVG in encoded64 string ?
if you would like to ONLY import the data, why did'nt you just use a Python Script to do this work for you ?
Python is my first choiche of a Tool when I would convert data for using it in FPC or an other DSL.

[Martin_fr]

There is indeed a speed issue (in SynEdit) for code with extremely long lines (depending on the hardware starting 100000 chars on one line, on most PC probably starting at between 250000 and 500000).

Part of this is the need to calculate the length for setting the scrollbar (or doing word wrap, if enabled / Lazarus 4.99 only).

And yes, doing the same amount of chars over multiple lines is faster. (not sure why)


---
If you don't use tabs and space trimming, you may get a little boost by removing those.

Something like

Code: Pascal  [Select][+][-]

1. x := SynEdit.TextviewManager.SynTextViewByClass[TSynEditStringTabExpander]; // TSynEditStringTrimmingList
2. SynEdit.TextviewManager.RemoveSynTextView(x);
3.  

If you do, make sure not to call stuff like "SynEdit.TabWidth" => that would then crash.

I haven't tested that ....

And if you have many lines, the tab view may even speed things up, as it has an internal cache. So you need to experiment.

[paule32]

on my Developer PC, mostly I use the Notepad++ Editor.
It can handle about 4 GiBytes of Code extremly fast, and with multiple Page Tabs.
So, I don't realy coded a own Editor - especialy when I only work on such kind of File
The Editor itself is Freeware (drop a Donut for the Work at the Authors :-)

[Martin_fr]

I know its possible to handle such large text.
But for now SynEdit has the issue with very long lines.

I don't know about ATSynEdit. Maybe it does better.

[Tommi]

on my Developer PC, mostly I use the Notepad++ Editor.
It can handle about 4 GiBytes of Code extremly fast, and with multiple Page Tabs.
So, I don't realy coded a own Editor - especialy when I only work on such kind of File
The Editor itself is Freeware (drop a Donut for the Work at the Authors :-)

Notepad++ isn't for Linux. Anyway I saw that it's based on Scintilla. Also Scite is based on Scintilla and it exists for Linux. So I tried it with a 100MB file and it works just like the file would be 1kB. It's very very quick. At the moment it's the only software under Linux with this capabilities.
Moreover it has a very permissive license:

Code: [Select]

Permission to use, copy, modify, and distribute this software and its
documentation for any purpose and without fee is hereby granted,
provided that the above copyright notice appear in all copies and that
both that copyright notice and this permission notice appear in
supporting documentation.
I am actually going to try to call Scintilla from Lazarus code.

Thank you