Recent

Author Topic: [Solved] Very slow assigning a TStringList to a TMemo  (Read 1139 times)

trev

  • Full Member
  • ***
  • Posts: 247
  • Former Delphi 7 and Delphi 10.2 User
[Solved] Very slow assigning a TStringList to a TMemo
« on: June 15, 2019, 09:14:50 am »
I'm assigning a TStringList to a TMemo - there are 4,004 strings of between 10 and 15 characters. On macOS this takes around 33 seconds and "beach balls" the cursor. Using Delphi, the macOS application takes around 2 seconds (I'm porting my application from Delphi to Lazarus).

Code: [Select]
Memo.Lines := strList;

I don't see any way of speeding this up...
« Last Edit: June 16, 2019, 01:31:02 am by trev »
o Lazarus v2.1.0 r61775, FPC v3.3.1 r42640, macOS 10.14.6 (with sup update), Xcode 10.3
o Lazarus v2.1.0 r61574, FPC v3.3.1 r42318, FreeBSD 12.0 (Parallels VM)
o Lazarus v2.1.0 r61574, FPC v3.0.4, Ubuntu 18.04 (Parallels VM)

SymbolicFrank

  • Hero Member
  • *****
  • Posts: 635
Re: Very slow assigning a TStringList to a TMemo
« Reply #1 on: June 15, 2019, 09:18:41 am »
Use TStrings.BeginUpdate and TStrings.EndUpdate. It turns off the display refresh.

trev

  • Full Member
  • ***
  • Posts: 247
  • Former Delphi 7 and Delphi 10.2 User
Re: Very slow assigning a TStringList to a TMemo
« Reply #2 on: June 15, 2019, 09:31:59 am »
Use TStrings.BeginUpdate and TStrings.EndUpdate. It turns off the display refresh.

There is no display. TStringList does not display; it is then assigned to Memo.Lines to avoid the delay of adding lines one by one to the Memo and also to sort the strings before adding.
« Last Edit: June 15, 2019, 09:45:38 am by trev »
o Lazarus v2.1.0 r61775, FPC v3.3.1 r42640, macOS 10.14.6 (with sup update), Xcode 10.3
o Lazarus v2.1.0 r61574, FPC v3.3.1 r42318, FreeBSD 12.0 (Parallels VM)
o Lazarus v2.1.0 r61574, FPC v3.0.4, Ubuntu 18.04 (Parallels VM)

Thaddy

  • Hero Member
  • *****
  • Posts: 9279
Re: Very slow assigning a TStringList to a TMemo
« Reply #3 on: June 15, 2019, 10:19:31 am »
Try memo.lines.text := stringlist.text, but plz do use beginupdate/endupdate. That's speed essential.
also related to equus asinus.

howardpc

  • Hero Member
  • *****
  • Posts: 3201
Re: Very slow assigning a TStringList to a TMemo
« Reply #4 on: June 15, 2019, 11:01:48 am »
Use TStrings.BeginUpdate and TStrings.EndUpdate. It turns off the display refresh.

There is no display. TStringList does not display; it is then assigned to Memo.Lines to avoid the delay of adding lines one by one to the Memo and also to sort the strings before adding.
There is a display. SymbolicFrank was writing about the Memo, not the strList.
Code: Pascal  [Select]
  1. procedure TForm1.Button1Click(Sender: TObject);
  2. var
  3.   i: Integer;
  4.   period: TTime;
  5.   ts: TTimeStamp;
  6. begin
  7.   strList := TStringList.Create;
  8.   try
  9.     for i := 0 to 4000 do
  10.       strList.Add('This is line number %d',[i]);
  11.     period := Time;
  12.     Memo1.Lines.BeginUpdate;
  13.     Memo1.Lines := strList;
  14.     Memo1.Lines.EndUpdate;
  15.     ts := DateTimeToTimeStamp(Time - period);
  16.     Memo1.Lines.Add('Operation took %d ms',[ts.Time]);
  17.     Memo1.SelStart := Length(Memo1.Text);
  18.   finally
  19.      strList.Free;
  20.   end;
  21. end;

trev

  • Full Member
  • ***
  • Posts: 247
  • Former Delphi 7 and Delphi 10.2 User
Re: Very slow assigning a TStringList to a TMemo
« Reply #5 on: June 15, 2019, 03:01:16 pm »
I did actually try it with the TMemo (and even the TStringList :) just in case.

Alas, no speed increase at all. See pic (I added your time code along with Begin/End around the Memo StringList assignment). The memo is not actually visible at the time its contents is added (clicking on the tabcontrol to show the memo causes it to read the file, sort the string, and then assign the memo).
« Last Edit: June 15, 2019, 03:05:28 pm by trev »
o Lazarus v2.1.0 r61775, FPC v3.3.1 r42640, macOS 10.14.6 (with sup update), Xcode 10.3
o Lazarus v2.1.0 r61574, FPC v3.3.1 r42318, FreeBSD 12.0 (Parallels VM)
o Lazarus v2.1.0 r61574, FPC v3.0.4, Ubuntu 18.04 (Parallels VM)

howardpc

  • Hero Member
  • *****
  • Posts: 3201
Re: Very slow assigning a TStringList to a TMemo
« Reply #6 on: June 15, 2019, 03:48:55 pm »
If you are convinced you are not timing the process of constructing the strList, only timing the process of assigning the pre-built strList to your memo's Lines, then I think that either the implementation of BeginUpdate/EndUpdate or of Lines.Assign() for TMemo on MacOS is flawed. I know nothing about the underlying widget used on the Mac for a memo, but your result is ridiculous.

Copying 4,000 strings from one memory location to another, and then displaying the last 20 or so, should take only a few ms, even on old hardware with long strings (yours are fairly short).

trev

  • Full Member
  • ***
  • Posts: 247
  • Former Delphi 7 and Delphi 10.2 User
Re: Very slow assigning a TStringList to a TMemo
« Reply #7 on: June 15, 2019, 03:57:24 pm »
Definitely convinced:

Code: Pascal  [Select]
  1.           strList.Sort;
  2.  
  3.           period := Time;
  4.           BitsMemo.Lines.BeginUpdate;
  5.           BitsMemo.Lines := strList;
  6.           BitsMemo.Lines.EndUpdate;
  7.           ts := DateTimeToTimeStamp(Time - period);
  8.           BitsMemo.Lines.Add('Operation took %d ms',[ts.Time]);
  9.  

The hardware is a 2018 Mac mini with six-core 3.2-4.6GHz i7, so not old or slow :)

I'll go pester the Cocoa widget magician tomorrow.
o Lazarus v2.1.0 r61775, FPC v3.3.1 r42640, macOS 10.14.6 (with sup update), Xcode 10.3
o Lazarus v2.1.0 r61574, FPC v3.3.1 r42318, FreeBSD 12.0 (Parallels VM)
o Lazarus v2.1.0 r61574, FPC v3.0.4, Ubuntu 18.04 (Parallels VM)

marcov

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 7592
Re: Very slow assigning a TStringList to a TMemo
« Reply #8 on: June 15, 2019, 04:05:02 pm »
The code looks ok.

I'd do a

 BitsMemo.Lines.Assign(strList);

but that is not the problem I think.

Note GUI widgets might not react very well to huge amounts of items, depending on widgetset and implementation.

ASerge

  • Hero Member
  • *****
  • Posts: 1420
Re: Very slow assigning a TStringList to a TMemo
« Reply #9 on: June 15, 2019, 09:41:38 pm »
In Windows the fastes way is Memo.Text := StringList.Text

trev

  • Full Member
  • ***
  • Posts: 247
  • Former Delphi 7 and Delphi 10.2 User
Re: Very slow assigning a TStringList to a TMemo
« Reply #10 on: June 16, 2019, 01:29:33 am »
In Windows the fastes way is Memo.Text := StringList.Text

Also in macOS  8-) Thankyou! See pic.

o Lazarus v2.1.0 r61775, FPC v3.3.1 r42640, macOS 10.14.6 (with sup update), Xcode 10.3
o Lazarus v2.1.0 r61574, FPC v3.3.1 r42318, FreeBSD 12.0 (Parallels VM)
o Lazarus v2.1.0 r61574, FPC v3.0.4, Ubuntu 18.04 (Parallels VM)

Thaddy

  • Hero Member
  • *****
  • Posts: 9279
Re: Very slow assigning a TStringList to a TMemo
« Reply #11 on: June 16, 2019, 06:02:05 am »
In Windows the fastes way is Memo.Text := StringList.Text
That's what I thought too, but note Marco's remark about assign. Internally the assignment := operator calls Assign, so that may be a little faster...
also related to equus asinus.

trev

  • Full Member
  • ***
  • Posts: 247
  • Former Delphi 7 and Delphi 10.2 User
Re: [Solved] Very slow assigning a TStringList to a TMemo
« Reply #12 on: June 16, 2019, 10:15:50 am »
For the record:

Code: [Select]
BitsMemo.Lines.Assign(strList);

Operation took 28067 ms
o Lazarus v2.1.0 r61775, FPC v3.3.1 r42640, macOS 10.14.6 (with sup update), Xcode 10.3
o Lazarus v2.1.0 r61574, FPC v3.3.1 r42318, FreeBSD 12.0 (Parallels VM)
o Lazarus v2.1.0 r61574, FPC v3.0.4, Ubuntu 18.04 (Parallels VM)

Thaddy

  • Hero Member
  • *****
  • Posts: 9279
Re: [Solved] Very slow assigning a TStringList to a TMemo
« Reply #13 on: June 16, 2019, 10:45:38 am »
That is unexpected? Anyway in that case my first hunch was right.
also related to equus asinus.

Remy Lebeau

  • Hero Member
  • *****
  • Posts: 680
    • Lebeau Software
Re: Very slow assigning a TStringList to a TMemo
« Reply #14 on: June 18, 2019, 10:43:20 pm »
Code: Pascal  [Select]
  1. Memo1.Lines.BeginUpdate;
  2. Memo1.Lines := strList;
  3. Memo1.Lines.EndUpdate;

I would expect the Lines property setter to call (Begin|End)Update() internally.  It does in Delphi (which simply calls FLines.Assign(), and then TStrings.Assign() handles the updating).

Code: Pascal  [Select]
  1. Memo1.SelStart := Length(Memo1.Text);

I would use the TMemo.GetTextLen() method instead of querying the Length of the Text property.  That way, it avoids having to actually read the full Text into memory first.

Code: Pascal  [Select]
  1. Memo1.SelStart := Memo1.GetTextLen;
Remy Lebeau
Lebeau Software - Owner, Developer
Internet Direct (Indy) - Admin, Developer (Support forum)