[SOLVED]Searching a term in Richmemo becomes very slow

[ariben]

Hi folks,
As I execute the following code the the Richmemo becomes very slow. Even the search term is only one.
Could anybody speed this up?

Code: Pascal  [Select][+][-]

1. procedure TForm2.DBGrid1DrawColumnCell(Sender: TObject; const Rect: TRect;
2.   DataCol: Integer; Column: TColumn; State: TGridDrawState);
3. var
4.   fp   :  TFontParams;
5.   PosSW1, PosSW2, PosSW3: Integer;
6.   i : Integer;
7.  begin
8.     fp.Color := clRed;
9.     fp.Style := [fsBold];
10.     fp.Size := 13;
11.     fp.BkColor := clWhite;
12.  
13.     if gdSelected in State then  //<---Everything was fine before added this!!!
14.     RichMemo1.Text:= Column.Field.AsString;            //RichMomo1に内容を表示する
15.  
16.     for i := 0 to UTF8Length(RichMemo1.Text) do                 //1語ずつケツ迄読んで
17.     begin                                                       //検索語を探す
18.     PosSW1 := RichMemo1.Search(SW1, i, UTF8Length(SW1), []);    //UTF8Length()を使うためには
19.     RichMemo1.SetTextAttributes(PosSW1, UTF8Length(SW1), fp);   //LazUtils, LazUTF8が必要
20.     if SW2 <> '' then
21.        begin
22.        PosSW2 := RichMemo1.Search(SW2, i, UTF8Length(SW2), []);
23.        RichMemo1.SetTextAttributes(PosSW2, UTF8Length(SW2), fp)
24.        end;
25.        if SW3 <> '' then
26.        begin
27.        PosSW3 := RichMemo1.Search(SW3, i, UTF8Length(SW3), []);
28.        RichMemo1.SetTextAttributes(PosSW3, UTF8Length(SW3), fp)
29.        end;
30.    end;
31. end;

[TRon]

disclaimer: I am not familiar with Richmemo

Your loop ...

Code: Pascal  [Select][+][-]

1. for i := 0 to UTF8Length(RichMemo1.Text) do
2.  

... Is more or less redundant because search already searches through all the richmemo contents (if I understood your search pattern correctly)

When you use the loop the search for SW1 is executed UTF8length(Richmemo1.text) times. That can literally take a very long time 

Imho a possible (not tested) option could be to replace that loop with something like

Code: Pascal  [Select][+][-]

1. posSW1 := 0;
2. while posSW1 < UTF8Len(Richmemo1.Text) and posSW1 <> -1 do
3. begin
4.   PosSW1 := RichMemo1.Search(SW1, PosSW1, UTF8Len(Richmemo1.Text) - PosSW1, []);
5.   // set attribute
6. end;
7.  
8. // similar loop for SW2
9.  
10. // similar loop for SW3
11.  
12.  

That is if I understood your code and the wiki correctly. Note that the snippet above does not take the length of SW1 into consideration (which can speed things up even further especially when the length of SW1 is considerable).

PS: I do not know where your richmemo component is situated but realize that DrawGridColumnCell can be invoked many times when scrolling your grid. Basically each time the gridcell needs to be repainted. Doesn't richmemo have an event that is triggered when you assign contents to it ? That would be a much more suitable location to set your attributes.

[TRon]

1. You do not seem to filter out the 'correct' record field (row) which, in case you show multiple rows at once (columns as well) means that every time the drawgridColumnCell
is triggered (which is for every visible and selected (see gdSelected in state) row/column) your search method(s) are executed.

For now, I leave that for/to yourself to address (and in case you want to).

2. the outer-loop (for i := 0 to UTF8Length(RichMemo1.Text) is unnecessary (as already mentioned before). Simply remove that, see also snippet posted below.

3. You are providing the length of your search term to the call to the richmemo search method (which is wrong, it should be the length of the content to search in , e.g. the length of the Memo contents (the earlier posted code-snipped showed that already). See earlier posted snippet and updated snippet below (which now also takes the length of the search term into consideration and skips it).

4. The loops with the Search and SetTextAttributes methods should be executed somewhere else. I tried to set the text using the text property but unfortunately that does not seem to trigger an onchange event. Also Update and invalidate where not able to trigger an OnChange Event so I have no clue which event to use to perform your search and highlighting routines.

The snippet below seem to work for me (except the aforementioned issues). I leave it up to you to address the length/UTF8len/UTF8Length mayhem's.

Code: Pascal  [Select][+][-]

1. procedure TForm1.PerformHighLight;
2. var
3.   fp          : TFontParams;
4.   posSW1      : integer = 0;
5.   SearchTerm1 : string = 'text to search for';
6. begin
7.   fp.Color := clRed;
8.   fp.Style := [fsBold];
9.   fp.Size := 13;
10.   fp.BkColor := clWhite;
11.  
12.   posSW1 := 0;
13.   while (posSW1 < RichMemo1.GetTextLen) and (posSW1 <> -1) do
14.   begin
15.     PosSW1 := RichMemo1.Search(SearchTerm1, PosSW1, RichMemo1.GetTextLen - PosSW1, []);
16.     if posSW1 > 0 then
17.     begin
18.       // set attribute
19.       RichMemo1.SetTextAttributes(PosSW1, Length(SearchTerm1), fp);
20.       // next search-position
21.       inc(PosSW1, Length(SearchTerm1));
22.     end;
23.   end;
24.  
25.   // repeat for SW2 and SW3
26. end;
27.  

[ariben]

Dear Tron,

Thank you very much for your code.  I was working almost all night long on this.
Your code works fine. But It's 7:00 in the morning in Japan so I need to go to my work place. Let me allow some more time to examine the detail.

And I didn't know the repaint of the grid so frequently. Now I know OnDrawColumn event is not appropriate for this action as you mentioned(as the window locks on this event).

Catch you later.

[TRon]

Thank you very much for your code.  I was working almost all night long on this.

Thank you for trying and confirming that it works for you.

Let me allow some more time to examine the detail.

No problem, take your time and in case of questions just ask. I don't know much about Richmemo but I have to figure it out myself eventually as well.

And I didn't know the repaint of the grid so frequently. Now I know OnDrawColumn event is not appropriate for this action as you mentioned(as the window locks on this event).

It depends entirely on your actual situation.

If you have a dbgrid with only one visible row or column then the event is triggered much less while having a full grid visible on your form calls the event for every cell it needs to (re)draw. The event at least allows to filter out by checking which field(name) is drawn. Usually the datasource is used to figure out which row is currently active but I can understand that your goal is to view the text (detailed in a richmemo) for the cell/row/column that is currently selected (by the user).

If you want to know how many time's the event is actually fired then place a temporary memo on your form next to your grid and append a line of text (preferably a counter) to that memo in the grid's DrawColumnCell event. It is a total waste of precious time to read the text from the database, assign it to a richmemo and perform highlighting for every time the event is fired. Your code needs to be performed only when the user changes from one row/column to another (if you have set it up that way) or when a change in the selection of a specific text field (in de grid) occurred.

Till next time 

[ariben]

Dear TRon and all,
Thank you very much for your code and great insights. I can never come up with such a nice code by myself now.
I translated your code into UTF8. As I commented I think I need an "=" in case that the content starts with the word searching for(? as far as I tried till tonight). As of now since the data is not so big, I'll go with the OnDrawColumnCell for the time being.
And UTF8LengthFast() does nice jobs, too.
I think I'm close to the goal.
Here is a reference to the people live in UTF8 who have relatively fewer informations.
Thank you very much again indeed.

Code: Pascal  [Select][+][-]

1. procedure TForm2.DBGrid1DrawColumnCell(Sender: TObject; const Rect: TRect;
2.    DataCol: Integer; Column: TColumn; State: TGridDrawState);
3.  var
4.  fp   :  TFontParams;
5.  PosSW1, PosSW2, PosSW3: Integer;
6.  //  i : Integer;
7.  begin
8.    fp.Color := clRed;
9.    fp.Style := [fsBold];
10.    fp.Size := 13;
11.    fp.BkColor := clWhite;
12.  
13.    PosSW1 := 0;
14.  //    //PosSW2 := 0;
15.  //    //PosSW3 := 0;
16. if gdSelected in State then begin
17.    RichMemo1.Text:= Column.Field.AsString;            //RichMomo1に内容を表示する
18.      while (PosSW1 < UTF8LengthFast(Richmemo1.Text)) and (PosSW1 <> -1) do begin
19.        PosSW1 := RichMemo1.Search(SW1, PosSW1, UTF8LengthFast(Richmemo1.Text) - PosSW1, []);
20.        if PosSW1 >= 0 then    // I think I need an "="
21.        begin
22.         RichMemo1.SetTextAttributes(PosSW1, UTF8LengthFast(SW1), fp);
23.        inc(PosSW1, UTF8LengthFast(SW1));
24.        end;
25.      end;
26.    end;
27. end;

I'd like make this as [SOLVED]

[TRon]

I translated your code into UTF8.

Great, thank you.

As I commented I think I need an "=" in case that the content starts with the word searching for

Yes indeed you are correct, it was my mistake (sorry for that)

As of now since the data is not so big, I'll go with the OnDrawColumnCell for the time being.

Just keep in mind that when things do become slow that you remember to look at this part. Make sure to comment it properly in your code 

And UTF8LengthFast() does nice jobs, too.

You could speed things up even further by calling UTF8LengthFast only once and store the result into an additional variable. You can do that for both UTF8LengthFast(Richmemo1.Text) and UTF8LengthFast(SW1).

The former does not change at all for any of the (3) searches and latter for each individual search.

Additionally you could opt to use an textlen variable for UTF8LengthFast(Richmemo1.Text) and subtract the utf8length of the searchterm from that and add one extra for one additional character. This because in case the number of characters left in the memo to search in is less then the length of the searchterm to search for then we already know we are not able to find the searchterm (anymore).

I'd like make this as [SOLVED]

Thank you for having done so and for reporting back.

Happy Coding !

[TRon]

Since it's almost Christmas again... a little Santa helper:

Code: Pascal  [Select][+][-]

1. type
2.   { TRichMemoHelper }
3.  
4.   TRichMemoHelper = class helper for TRichMemo
5.     function HighLightStrings(const aStrings: array of string): integer;
6.   end;
7.  
8. { TRichMemoHelper }
9.  
10. function TRichMemoHelper.HighLightStrings(const aStrings: array of string): integer;
11. var
12.   fontparams          : TFontParams;
13.   current_position    : integer;
14.   last_position       : integer;
15.   text_length         : integer;
16.   searchstring_length : integer;
17.   searchstring        : string;
18.   number_of_matched_highlights : integer absolute result;
19. begin
20.   // no matches yet
21.   number_of_matched_highlights := 0;
22.   // how longwinded is the text to search in (1 * UTF8Len)
23.   text_length := UTF8Length(Self.Text);
24.  
25.   for searchstring in aStrings do
26.   begin
27.     // ( Length(aStrings) * UTF8Len)
28.     searchstring_length := UTF8Length(searchstring);
29.     // ( Length(aStrings) * calculation)
30.     last_position       := text_length - searchstring_length + 1;
31.     // start from beginning of text to search in
32.     current_position    := 0;
33.  
34.     // keep searching until a) end of text or b) searchstring not found
35.     while (current_position <> -1) and (current_position < last_position) do
36.     begin
37.       // perform actual search
38.       current_position := Self.Search(searchstring, current_position, last_position - current_position, []);
39.       // located a match
40.       if current_position >= 0 then
41.       begin
42.         // keep track of number of found matches (= highlights performed)
43.         inc(number_of_matched_highlights);
44.         // get current fontparams
45.         Self.GetTextAttributes(current_position, fontparams);
46.         // change appearance of matched searchstring
47.         fontparams.Color := clRed;
48.         fontparams.Style := [fsBold];
49.         Self.SetTextAttributes(current_position, searchstring_length, fontparams);
50.         // next position in text
51.         inc(current_position, searchstring_length);
52.       end;
53.     end;
54.   end;
55. end;
56.  

usage:

Code: Text  [Select][+][-]

1. ]
2. procedure TForm1.Button1Click(Sender: TObject);
3. var
4.   Lines: TStringList;
5.   n: integer;
6. begin
7.   Lines := TStringList.Create;
8.   try
9.     Lines.LoadFromFile('gpl-3.0.txt');
10.     RichMemo1.Text := Lines.Text;
11.     n := RichMemo1.HighLightStrings(['warranty', 'license']);
12.     Memo1.Append('number of performed highlights = ' + n.ToString);
13.   finally
14.     Lines.Free;
15.   end;
16. end;
17.  

[ariben]

Wow, How cool it is !!
You have not only taken the next task of mine but also taken my breath away!
It's February, though I realize Santa does exist.
I was just thinking how to solve this daunting project. I don't know the best reward for you but saying thank you very much.
To be honest, I have no idea how, when and where to use "Self" as well as what it means. I'll check out my text book again and want to get along with it.
And....

Just keep in mind that when things do become slow that you remember to look at this part. Make sure to comment it properly in your code

Sure thing. I remind my successors of this.

You could speed things up even further by calling UTF8LengthFast only once and store the result into an additional variable. You can do that for both UTF8LengthFast(Richmemo1.Text) and UTF8LengthFast(SW1).

The former does not change at all for any of the (3) searches and latter for each individual search.

Additionally you could opt to use an textlen variable for UTF8LengthFast(Richmemo1.Text) and subtract the utf8length of the searchterm from that and add one extra for one additional character. This because in case the number of characters left in the memo to search in is less then the length of the searchterm to search for then we already know we are not able to find the searchterm (anymore).

But you are done before I understand. Now I say thank you again from the bottom of my heart.

[TRon]

I was just thinking how to solve this daunting project. I don't know the best reward for you but saying thank you very much.

You are more than welcome.

You can reward me by learning from my bad coding practices and pass along the (positive) knowledge of what you've learned. IOW Pay it forward my friend ! 

Just try to play with the code, try to extend and mold it to your liking.

To be honest, I have no idea how, when and where to use "Self" as well as what it means. I'll check out my text book again and want to get along with it.

I could have omitted the use of Self as it refers to the instance of the object (in this case the instance of the richmemo.

The helper is declared as a helper for class richmemo so Self refers to richmemo).

The benefit of using Self is that you can use the Lazarus editor with its code-tools and press ctrl-space right after the dot of having typed Self (which in itself is context sensitive, f.e. inside a form method Self will refer to the form) and codetools will automatically pop-up all the methods, events and properties of Self (TRichMemo) in a small window and, after selecting the right property/method/event from the pop-up window and pressing enter codetools automatically paste the name into the editor and provides hints when you try to add parameters to the just pasted function/method/property-name.

But you are done before I understand. Now I say thank you again from the bottom of my heart.

Yes, I realize. Hence Christmas is early this year 

Oh, one disclaimer: It might be the code is one off somewhere... 

Happy coding !