[solved] slow implementation of TStrings.Delete method?

[kegge13]

To limit the total length of a memo which receives logging messages during prolongated use I added:

Code: Pascal  [Select][+][-]

1. while LogTabMemo.Lines.Count>LogTabMemo.Tag do
2.   LogTabMemo.Lines.Delete(0);
3.  

When I set the tag value to 500 and write some 900 lines (in total some 200 kB) this takes about a minute.
In Delphi7 this gives no significant delay at all.

Setting the Tag value above the total number of lines written, fills the memo without any delay.

Is there an explanation and/or a remedy?

Thanks in advance.

[wp]

With every Delete there is a repaint... To avoid this, you must put your code into a BeginUpdate/EndUpdate block:

Code: Pascal  [Select][+][-]

1. LogTabMemo.Lines.BeginUpdate;
2. try
3.   while LogTabMemo.Lines.Count>LogTabMemo.Tag do
4.     LogTabMemo.Lines.Delete(0);
5. finally
6.   LogTabMemo.Lines.EndUpdate;
7. end;

[kegge13]

Thanks for the suggestion.  I placed most of the code that fills the memo inbetween the BeginUpdate and the EndUpdate.
No improvement although. What does work it to just clear all contents (Lines.Clear). That is very fast again.

[ASerge]

No improvement although.

Code: Pascal  [Select][+][-]

1. var
2.   L: TStringList;
3. begin
4.   L := TStringList.Create;
5.   try
6.     L.Text := Memo1.Text;
7.     L.Capacity := 500;
8.     Memo1.Text := L.Text;
9.   finally
10.     L.Free;
11.   end;
12. end;

[marcov]

I assume you are using windows?

[wp]

It's hard to believe that loading 900 strings into a TMemo and then deleting 400 to keep 500 lines would take as long as a minute.

But you are right. I am attaching a little demo with two buttons: the first one adds 1000 random strings to a memo, the second one deletes always the first string as long as 900 strings are left. Although in this example only 100 strings are deleted it takes 12 seconds on my pc (Win 10).

Debugging into the Delete(0) leads me into the widget set, and for Windows I see this code:

Code: Pascal  [Select][+][-]

1. procedure TWin32MemoStrings.Delete(Index: integer);
2. var
3.   LineStart,
4.   LineEnd: Integer;
5. begin
6.   LineStart := GetLineStart(Index);
7.   LineEnd := GetLineStart(Index+1);
8.   if LineEnd < 0 then LineEnd := LineStart+GetLineLength(Index);
9.   SendMessageW(FHandle, EM_SETSEL, LineStart, LineEnd);
10.   SendMessageW(FHandle, EM_REPLACESEL,0 , lparam(PWChar('')));
11. end;

The SendMessageW instructions communicate with the operating system, this is because the strings are not stored by the memo, but by the widgetset. This probably is the explanation why the Deleting process is so awfully slow.

However, when I bypass the widgetset by first copying the memo lines to a temporary stringlist, deleting the strings in the stringlist and copying the remaining strings back to the memo, the process is as fast as expected.

I get the same behavior when I run the same project in Delphi XE10.3. So, are you really doing the same when you make the observation that delphi is much faster than Lazarus here?

[440bx]

the first one adds 1000 random strings to a memo, the second one deletes always the first string as long as 900 strings are left. Although in this example only 100 strings are deleted it takes 12 seconds on my pc (Win 10).

Just FYI, a number of Windows controls do a simple linear search to locate an element.  In addition to that, some use a singly linked list which causes deletions to be time consuming.  They are ok only for a limited number of elements, ballpark of a couple hundred elements but, beyond that, their performance is iffy.

Sounds like the memo control is one of those.

[jamie]

A memo gets over used. It was never intended for large listings.

 I've noticed some time ago that MS started to degrade the performance of certain API and control functions. Some API's are the old 16 bit ones supported in the 32 bit world. Those I've found MS has made them extremely slow.. They work but slow..

 Basically a deterrent to force code updates. There is no reason for the slowness of the TMEMO since it used to operate much faster on large list, now it does not..

 But getting back to what  I was originally going to say, the app should be managing large list of data and use these so called MEMO's for short listings..

 In your code you can log to a TStringList and maintain some last line indexes in code somewhere and during a screen up simply assign only a few lines at a time to the memo to view them... Don't load the memo to the hills.

[Remy Lebeau]

Code: Pascal  [Select][+][-]

1. var
2.   L: TStringList;
3. begin
4.   L := TStringList.Create;
5.   try
6.     L.Text := Memo1.Text;
7.     L.Capacity := 500;
8.     Memo1.Text := L.Text;
9.   finally
10.     L.Free;
11.   end;
12. end;

Wouldn't Assign() be better?

Code: Pascal  [Select][+][-]

1. var
2.   L: TStringList;
3. begin
4.   L := TStringList.Create;
5.   try
6.     L.Assign(Memo1.Lines);
7.     L.Capacity := 500;
8.     Memo1.Lines.Assign(L);
9.   finally
10.     L.Free;
11.   end;
12. end;

Debugging into the Delete(0) leads me into the widget set, and for Windows I see this code:

Code: Pascal  [Select][+][-]

1. procedure TWin32MemoStrings.Delete(Index: integer);
2. var
3.   LineStart,
4.   LineEnd: Integer;
5. begin
6.   LineStart := GetLineStart(Index);
7.   LineEnd := GetLineStart(Index+1);
8.   if LineEnd < 0 then LineEnd := LineStart+GetLineLength(Index);
9.   SendMessageW(FHandle, EM_SETSEL, LineStart, LineEnd);
10.   SendMessageW(FHandle, EM_REPLACESEL,0 , lparam(PWChar('')));
11. end;

That is roughly the same as how Delphi removes a line from a VCL TMemo:

Code: Pascal  [Select][+][-]

1. procedure TMemoStrings.Delete(Index: Integer);
2. var
3.   SelStart, SelEnd: Integer;
4. begin
5.   SelStart := SendMessage(Memo.Handle, EM_LINEINDEX, Index, 0);
6.   if SelStart >= 0 then
7.   begin
8.     SelEnd := SendMessage(Memo.Handle, EM_LINEINDEX, Index + 1, 0);
9.     if SelEnd < 0 then SelEnd := SelStart +
10.       SendMessage(Memo.Handle, EM_LINELENGTH, SelStart, 0);
11.     SendMessage(Memo.Handle, EM_SETSEL, SelStart, SelEnd);
12.     SendTextMessage(Memo.Handle, EM_REPLACESEL, 0, '');
13.   end;
14. end;

[winni]

Hi!

Don't let the Memo.lines grow up to 900 or more lines:

After every Memo.lines.add execute your above shown code.
Then you will only delete one lines which you will not notice.

And the name of the TMemo shows for what to use: Memos!
Otherwise it would be called TLog.

Winni

[marcov]

Hi!

Don't let the Memo.lines grow up to 900 or more lines:

After every Memo.lines.add execute your above shown code.
Then you will only delete one lines which you will not notice.

(and that is a problem in D7 too, as I found out to my detriment once  Forgetting to regularly to clear a log can cause really funky behaviour, specially if your app is at least somewhat realtime)

[jamie]

for a quick runner you could use SynEdit but what ever..

For a more elegant way of doing it without writing a control is to create a TFrame with a TStringlist and managed the display surface on updates..

 this way you can put that in your archives for later use as a Logger or what ever..

[winni]

(and that is a problem in D7 too, as I found out to my detriment once  Forgetting to regularly to clear a log can cause really funky behaviour, specially if your app is at least somewhat realtime)

Borland was so proud that a Memo in Delpi2 (32 bit!) could contain more than 32767 byte - as opposed to Delphi1 (16 bit).
But if you had more than 2^15 byte in the stringlist things started to become strange.
In those days we thought that it had to do with our small some Megabyte RAM.
But it was obviously a hidden bug.

I don't know if the have fixed it until now: I don't own an actual Delphi.

Winni

[jamie]

I remember using TP 6.0 and I believe this error migrated to Delphi 1, too..

IF a source file got to a particular size, somewhere around a page boundary,  you would get a compiler error indicating something wrong with your code but there wasn't anything wrong with the code at that section. looking at the disk size of the code on file indicated something like 8192 or there abouts.

 The fix was to insert a comment or move things around so there was no code in that section of the file and then it would compile error free.
 
 Just think of parameters that were getting ghosted but passing the compile test when hitting that multiple error point.. That is what you call a real hard to fine bug.

[ASerge]

Wouldn't Assign() be better?

No. View the Assign code. For visual control, it is faster to get/set the entire text.