Recent

Author Topic: [SOLVED] 2.2 win x64: ugly issue with virtual listview "Selected" performance  (Read 13730 times)

d7_2_laz

  • Hero Member
  • *****
  • Posts: 510
First of all: after recompilaton of two bigger progs prviously ported from Delphi 7 with 2.2 RC2 Win x64 i encountered nearly no problems. Many compliments to the developers here, that is impressive! And many thanks from my side!

Except those two issues with the virtual listview that, introduced with RC1, - unfortunately - still do not have any status at all.

Those issues do persist:

(1) Virtual listview, MultiSelect: early reference to Selected object may cause performace problems
https://gitlab.com/freepascal.org/lazarus/lazarus/-/issues/39324
-> This issue kills the performance of my prog completely, it breaks long existing code without any need. Only solution so far is a workaround.
Remark: within the test project, the guilty statemant might appear to be very obvious. I intended that of course. But, how to determine what is a good place and what a wrong place?
In reality, the reference to "Selected" or similar within some subordinary routines might be somewhere in the code. And so it might be very hard to find out why all slows down.
A simple "if Selected" at the wrong place, and .. boom, the virtual listview behaves just the contrary as "virtual", as it immediately instantiates all objects at once. The culprit may be deep in the code. No fun at all to rewrite parts of the app only due to this shortly arriving issue.

(2) Virtual listview, MultiSelect: problem: only one item in the list: after click on empty space this single item keeps to stay selected
https://gitlab.com/freepascal.org/lazarus/lazarus/-/issues/39325
-> This issue results in wrong effects when doing owner drawing. Items may appear to be selected ... although they aren't.
Remark: for a solution, i'd recommend to avoid expensive loops around all items for to do a correction of the property.

I would be very happy to see at least somwthing like a status on the issues. Would this be possible?
For information i attach my workaround (based on RC2) here that lets the program behave as expected again.

The good news is: except those two ugly beasts no problems at all with RC2 so far!
« Last Edit: January 28, 2022, 08:47:33 pm by d7_2_laz »
Lazarus 3.2  FPC 3.2.2 Win10 64bit

d7_2_laz

  • Hero Member
  • *****
  • Posts: 510
2.2 win x64: two persistent ugly issues with virtual listview "Selected"
« Reply #1 on: January 15, 2022, 01:07:15 pm »
Best congratulations for the excellent 2.2! (i like to wokr with Lazarus a lot)
Nearly no problems at all  ... except those remaining two (win x64):

No change here with 2.2, same as newly introduced with RC1 and RC2:

Virtual listview, MultiSelect: early reference to Selected object may cause performance problems
https://gitlab.com/freepascal.org/lazarus/lazarus/-/issues/39324

The only thing i'd like to have is to be downward-compatible about basic objects and properties.
So that one can refer to "Selected" (or "SelCount") anywhere within the code.
Without time-consuming loops around all items - and without (and this is even more worse:) all items of the list are unintendedly instantiated at once bulk-wise!

Code: Pascal  [Select][+][-]
  1.   ListView1.Items.Count := 20000;
  2.   // Probably somewhere later, but for demo in its elementary form here:
  3.   if ListView1.Selected = nil then   -> boom! 20000 items will be instantiated at once, destroying virtualism
  4.   // Just saw that it is the same with "SelCount":
  5.   if ListView1.SelCount > 0 then     -> boom! all 20000 items will be instantiated at once

"Ugly", because the instantiating "Selected" (or "SelCount") might be located anywhere.
For instance in some multi-called helper routines. Hard to find out what's going on here and old code breaks! Initially it did drive me crazy
until i found out what caused the drastic performance decrease in my app.
Remark: the windows folder "C:\Windows\WinSxS" does contain nearly 20000 items. that was causing my observation.
Question: how to decide under which circumstances one can refer to
"Selected" or "SelCount" without harm??

The problem did not exist with 2.0.12 and before, but with 2.2 RC1, 2.2 RC2 and 2.2.

Virtual listview, MultiSelect: problem: having only one item in the list: after click on empty space this single item keeps to stay selected
https://gitlab.com/freepascal.org/lazarus/lazarus/-/issues/39325

After clicking on empty area, it should be possible to rely on the last item correctly being de-selected.
For instance for to achieve correct owner-drawing if one use that for specific paint of selected objects!
Imo, the evaluation should be done best without time-expensive loops along all 50000 items.

I'd highly wish to see, after four months, at least something like a status resp. label on these issues so that there is an indication that they don't will get forgotten.
Lazarus 3.2  FPC 3.2.2 Win10 64bit

ASerge

  • Hero Member
  • *****
  • Posts: 2222
Re: 2.2 win x64: two persistent ugly issues with virtual listview "Selected"
« Reply #2 on: January 15, 2022, 02:09:10 pm »
Code: Pascal  [Select][+][-]
  1.   ListView1.Items.Count := 20000;
  2.   // Probably somewhere later, but for demo in its elementary form here:
  3.   if ListView1.Selected = nil then   -> boom! 20000 items will be instantiated at once, destroying virtualism
  4.   // Just saw that it is the same with "SelCount":
  5.   if ListView1.SelCount > 0 then     -> boom! all 20000 items will be instantiated at once
Windows 7 x64, Lazarus 2.2.0
Form with ListView (OwnerData=True, ViewStyle=vsReport, 2 columns), Memo and two buttons:
Code: Pascal  [Select][+][-]
  1. procedure TForm1.Button1Click(Sender: TObject);
  2. begin
  3.   ListView1.Items.Count := 20000;
  4. end;
  5.  
  6. procedure TForm1.Button2Click(Sender: TObject);
  7. begin
  8.   if ListView1.Selected = nil then
  9.     Memo1.Append('nil');
  10. end;
  11.  
  12. procedure TForm1.ListView1Data(Sender: TObject; Item: TListItem);
  13. var
  14.   S: string;
  15. begin
  16.   S := IntToStr(Item.Index);
  17.   Memo1.Append(S); // Logging of requested items
  18.   Item.Caption := S;
  19.   Item.SubItems.Append(S + ' ' + S);
  20. end;
After pressing Button1, only visible elements are requested. After pressing Button2 - not one at all.

d7_2_laz

  • Hero Member
  • *****
  • Posts: 510
Re: 2.2 win x64: two persistent ugly issues with virtual listview "Selected"
« Reply #3 on: January 15, 2022, 02:50:49 pm »
Hello ASerge,

many thanks for your reply!
Unfortunately that's not the point.
Could you
- set the ListView to "MultiSelect" (problem only here)
- and move your lines 8 and 9 (if ListView1.Selected = nil  etc.) directly after line3?    (but this is meant only as a simple demo)
What do you see how many items are initialized at once?

My testcases are placed within the issue reports.
Lazarus 3.2  FPC 3.2.2 Win10 64bit

ASerge

  • Hero Member
  • *****
  • Posts: 2222
Re: 2.2 win x64: two persistent ugly issues with virtual listview "Selected"
« Reply #4 on: January 15, 2022, 03:22:46 pm »
Could you
- set the ListView to "MultiSelect" (problem only here)
You're right. After that, when press Button 2, all the elements are requested.
But if I change (ListView1.Selected = nil) to (ListView1.SelCount = 0) no items requested. In this case, a call is made via the widget without iterating over the elements.

d7_2_laz

  • Hero Member
  • *****
  • Posts: 510
Re: 2.2 win x64: two persistent ugly issues with virtual listview "Selected"
« Reply #5 on: January 15, 2022, 03:42:58 pm »
You are right about the SelCount (but that was only an observation besides).
But if you move the assignment to Items.Count and the "if ListView1.SelCount" etc both from within Button1Click into FormCreate, it will happen too (strange).
Code: Pascal  [Select][+][-]
  1. procedure TForm1.FormCreate(Sender: TObject);
  2. begin
  3.    ListView1.Items.Count := 20000;
  4.    if ListView1.SelCount = 0  then
Lazarus 3.2  FPC 3.2.2 Win10 64bit

ASerge

  • Hero Member
  • *****
  • Posts: 2222
Re: 2.2 win x64: two persistent ugly issues with virtual listview "Selected"
« Reply #6 on: January 16, 2022, 11:10:45 am »
You are right about the SelCount (but that was only an observation besides).
But if you move the assignment to Items.Count and the "if ListView1.SelCount" etc both from within Button1Click into FormCreate, it will happen too (strange).
Since the ListView1.Handle has not been created, Lazarus does not call the OS function.

d7_2_laz

  • Hero Member
  • *****
  • Posts: 510
Re: 2.2 win x64: two persistent ugly issues with virtual listview "Selected"
« Reply #7 on: January 16, 2022, 02:45:50 pm »
Hm, right, see TCustomListView.GetSelCount: If not HandleAllocated, then it gooes into a loop with .. guess what? Querying if an item is Selected.

So we're back here at the main issue (with "Selected").

Unintentionally it's an illusration why i said this issue is so treacherous: because the all-data-requesting "Selected" might be located anywhere within the code hierarchy. One never can be sure ...
Lazarus 3.2  FPC 3.2.2 Win10 64bit

d7_2_laz

  • Hero Member
  • *****
  • Posts: 510
Re: 2.2 win x64: two persistent ugly issues with virtual listview "Selected"
« Reply #8 on: January 20, 2022, 05:44:58 pm »
Sorry to bother again here, but please allow a small supplementary note.
It's only because i had been quite unhappy myself about my own test case. Because it's restricted as much to it's most minimalistic form.
One might think: this won't affect me, never, i won't do such! To ask for "if Selected .." immediately after having set Items.Count  Crazy!

So i tried an at least b bit more "realistic" use case (attached):
- press button 1 for to do some initialization; here: set Item.Count
- press button 2 for to see later and separately if a selection had been done in the listview.
When no selection happened, you can see:

Listview Item.Count had been set to  30000
No item selected
That took 547 ms
===>>>>    # Calls of OnData had been:  30013
Lazarus 3.2  FPC 3.2.2 Win10 64bit

balazsszekely

  • Guest
Re: 2.2 win x64: two persistent ugly issues with virtual listview "Selected"
« Reply #9 on: January 20, 2022, 07:09:30 pm »
@ d7_2_laz
VST use a dynamic array to keep a list with selected items. ListView on the other hand, loops through the items and checks each item selected state, no wonder is slow. However in case of owner draw data, you are lucky, because TCustomListView internally keeps track the index of the selected item. So when nothing is selected, you can avoid the loop. Try attached patch against trunk and see if it solves the problem.

d7_2_laz

  • Hero Member
  • *****
  • Posts: 510
Re: 2.2 win x64: two persistent ugly issues with virtual listview "Selected"
« Reply #10 on: January 20, 2022, 07:57:20 pm »
Hi Getmen,
thank you!
Could apply it directy to my 2.2.-  If i understand the patch file format right,
it's simply about to apply this additional  if-condition before the for loop.
Works very fine! Btw it's essentially the same (but shorter coded)  than my workaround proposed:

Code: Pascal  [Select][+][-]
  1.       if OwnerData And (FSelectedIdx=-1) then begin
  2.          Result := nil;
  3.          FSelected := nil;
  4.          Exit;
  5.       end;
  6.       FSelected := nil;
  7.       for i := 0 to Items.Count - 1 do
  8.      //  etc. .....

The patch would be fine for me  :)
Lazarus 3.2  FPC 3.2.2 Win10 64bit

balazsszekely

  • Guest
Re: 2.2 win x64: two persistent ugly issues with virtual listview "Selected"
« Reply #11 on: January 20, 2022, 08:22:11 pm »
@d7_2_laz
Quote
Could apply it directy to my 2.2.-  If i understand the patch file format right,
it's simply about to apply this additional  if-condition before the for loop.
Works very fine!
You're welcome! In my opinion the chance of regression for this patch is minimal.

Quote
Btw it's essentially the same (but shorter coded)  than my workaround proposed
Where do you proposed a workaround? In the bug tracker?

d7_2_laz

  • Hero Member
  • *****
  • Posts: 510
Re: 2.2 win x64: two persistent ugly issues with virtual listview "Selected"
« Reply #12 on: January 20, 2022, 08:54:43 pm »
Yes GetMem, here (lthe link is mentioned at the begin of this post)
https://gitlab.com/freepascal.org/lazarus/lazarus/-/issues/39324

It's based (and slightly modified) on a proposal from yourself  around July 28, 2021

Since this time i'm trying to see this issue fixed officiially, until now without any success   :o
Endless story, see https://forum.lazarus.freepascal.org/index.php/topic,55398.0.html

It killed my app in at least 3 different respects.  Whereas i could use the workaround without any problems.
Lazarus 3.2  FPC 3.2.2 Win10 64bit

dsiders

  • Hero Member
  • *****
  • Posts: 1052
Re: 2.2 win x64: two persistent ugly issues with virtual listview "Selected"
« Reply #13 on: January 20, 2022, 09:11:55 pm »
It killed my app in at least 3 different respects.  Whereas i could use the workaround without any problems.

Are you saying it passes the test case Martin mentioned in the post? If so, please add that comment to the bug report,
Preview Lazarus 3.99 documentation at: https://dsiders.gitlab.io/lazdocsnext

balazsszekely

  • Guest
Re: 2.2 win x64: two persistent ugly issues with virtual listview "Selected"
« Reply #14 on: January 20, 2022, 09:13:53 pm »
Quote
It's based (and slightly modified) on a proposal from yourself  around July 28, 2021
Apparently the first bad sectors already appeared in my head.  :D

My original proposal was wrong. The new patch from my previous post should be fine. Can you please test if it has side effects?
In my opinion should not cause any issues. I will test your project with gtk2.

PS: The patch works fine with gtk2(Linux Mint 20.02 Cinnamon). As a side not the listview is much faster on linux then on windows.
« Last Edit: January 20, 2022, 09:26:33 pm by GetMem »

 

TinyPortal © 2005-2018