Filter improvments for "Search Results" window / and others

[Martin_fr]

I will also improve the filter - it will be possible to filter not only text, but also files and paths. But I'll deal with that a bit later.

In my 2 cents imagination I see 2 edit fields - one include filter and one exclude filter. Filtering keywords in edit field are separated by spaces, and when complex they can be put in quotes. Edit fields can be permanent or shown only after pressing "+" and/or "-" captioned buttons. And filter is live and incremental.

Please don't shoot me for dreaming big without having my own hands dirty 

Why 2 edits?

I recently saw https://gitlab.com/freepascal.org/lazarus/lazarus/-/issues/40245#multi-word-search  (unfold the section)

- It has ! for terms that must not be present.
- " are a bit awkward, as they are often used for "words that must be in the given sequence".
- And for the filter in search results there need to be an escape, since we may filter for any of the control chars


Filter edit could have properties to configure this (including which separator chars, and defaults).
The IDE could then adapt those search patterns in all filters. (and maybe even as an "extended" syntax in the search dialog)

Maybe (and adapt the linked filter too)

Code: Pascal  [Select][+][-]

1. property FilterEdit.MultiWord = (mwMatchAll, mwMatchAsSentence, mwMatchAnyOne)
2. property FilterEdit.EscapeChar = '\'
3. property FilterEdit.MarkersRequireWhiteSpace = True;  // abc&def is not a & marker
4.  
5.  
6. SentenceMarker = '"' // Encloses Sentence
7. // Not sure if we need a separate Begin/End >Foo Bar<
8.  
9. MatchAnd = ['&', '+' {, '\and'}]  // foo & bar \and xyz
10. MatchOr = ['|', ',' {, '\or'}  // foo | bar \or xyz
11. MatchNot = ['!', '-' {, '\not'}  // foo | bar \or xyz
12. MatchWordBound = '*' // or %~
13. MatchWordStart = '^'
14. MatchWordEnd = '$'
15.  
16. UseBrackets = (bNone, bRound, bRoundEscaped, bSquare, bSquareEscaped)
17.  

[n7800]

I recently saw https://gitlab.com/freepascal.org/lazarus/lazarus/-/issues/40245#multi-word-search  (unfold the section)

Yes, that's exactly what I want to adapt for these purposes. But right now it only works for identificators (which can only contain ASCII, or rather, only letters, numbers and underscores).
And absolutely any characters can be found in the search result view window. So I will need to add UTF8 support, escaping, and the ability to filter by files. Some of this is already ready for me, but can we discuss it in a separate topic?

[Martin_fr]

I recently saw https://gitlab.com/freepascal.org/lazarus/lazarus/-/issues/40245#multi-word-search  (unfold the section)

can we discuss it in a separate topic?

Yes. / Moved

[n7800]

About "Search results window":
To begin with, I note that the user must enable the multi-word search himself, otherwise many users will be surprised that the search "stopped working" as usual. So we need another button or option...

[n7800]

I have added MultiWordSearch to several windows already. I believe that it should work everywhere the same, so that it is convenient for the user to use it. Therefore, it is desirable to make the search in this window similar. That is, we just need to try to expand it.

[n7800]

To the existing symbols, I suggest using the following:
' - quotation marks (this is usual for Pascal programmers - all string constants are enclosed in such quotation marks).
'' - to escape the quote character itself (also from Pascal).

It turns out that any text can simply be taken into 'text' and everything will work in the old way.

This, by the way, will allow you to continue using double quotes to search only at the beginning, end, or entire phrase. I would not like to remove this feature. I don't think a Pascal programmer would confuse single quotes with double quotes.

[n7800]

But it is desirable to replace the "," sign in this window with "|" (logical OR), since the comma is often found in the code. I think the same can be done in MultiWordSearch itself too.

[n7800]

To search within a path, I suggest using the "@" sign, like so:

@\main.pp		search in "main.pp" file
!@\backup\		do not search in "backup" folder
@.pp"		show only file with ".pp" extension

[n7800]

Filter edit could have properties to configure this (including which separator chars, and defaults).
The IDE could then adapt those search patterns in all filters. (and maybe even as an "extended" syntax in the search dialog)

Yes, I wanted to suggest something similar, at least for TTreeFilterEdit and TListFilterEdit.

As well as the possibility of "auto-transliteration" (see in the same issue http://gitlab.com/freepascal.org/lazarus/lazarus/-/issues/40245), as a boolean property.
It may have a name like: "AllowOnlyASCII" or "OnlyQWERTY".

[Martin_fr]

With so many "change-able" behaviours, we do need an option page (in Tools Options).
Most people will want there own mix of defaults, and not having to toggle them all the time.

Or at least sticky settings.

the user must enable the multi-word search himself, otherwise many users will be surprised that the search "stopped working" as usual. So we need another button or option

Not a button (well maybe a dropdown, see TortoiseGit) ... Maybe make it a context menu, on the filter edit. Well ok, that is for the copy/paste/... stuff, but it could have the rest in there.
See image how it works in TortoiseGit

Though... this is for filters everywhere. FilterEdit are used throughout the IDE. And they should offer all the same behaviour.
Therefore this needs to be an option.

If it then can be toggled away from the option, then should that
- affect the one filter only (persistent, or for one search)?
- affect all filters

If the mode can be changed, some way to see the current mode would be nice (background image?)

I have added MultiWordSearch to several windows already. I believe that it should work everywhere the same, so that it is convenient for the user to use it. Therefore, it is desirable to make the search in this window similar. That is, we just need to try to expand it.

It should definitely the same everywhere.

Just need to define what it should be everywhere. The start you made is great, but lets get all the ideas together.
And also ensure that later additions can be made easily to affect all the windows....

Btw: "Debug Inspector" has a filter (but not yet a filter edit). It could benefit too.
And watches/locals will eventually get it too.

Here there will also be search in different "columns": variable-name/expression  ; value ; address;  ...

' - quotation marks (this is usual for Pascal programmers - all string constants are enclosed in such quotation marks).
'' - to escape the quote character itself (also from Pascal).

Ok.

I still think alternatively / optional escaping with \ is a good idea too. Many people are used to that too (like in regex). And if we have global options, then it is easy to toggle.

will allow you to continue using double quotes to search only at the beginning, end, or entire phrase.

Ok.
Btw, they alse need to be escape-able. And I am not that sure about: ""  (and then also how to escape the coma , ? Or the space for that matter)

Ok escaping space (and coma) isn't needed, if you put all into single quotes. But if you modify a search term it may be less work to type \, than to navigate to the begin of the current sub-term and insert the opening `

But it is desirable to replace the "," sign in this window with "|"

+1
Correct above for | instead of ,

To search within a path, I suggest using the "@" sign, like so:

I do like the idea of having a "search operator" for that.

But see my comment about watches. There are several columns.

And also in the search result, you have the following options:
- search all
- search only found text
- search only path
- search only line-num
- search all, except: ...

If we change it to

@: search only path
@t: search only found Text
@!t: search except found Text
@tl: text and line
@p: Path (if omitted "p" is default), hence @: works
@!: all except path

Its a bit complex, better ideas welcome.

Yes, I wanted to suggest something similar, at least for TTreeFilterEdit and TListFilterEdit.

Shouldn't it be in the base class ?

It may have a name like: "AllowOnlyASCII" or "OnlyQWERTY".

Or should it be a an enum?
- Only Latin
- Only digits (that may be of interest(
- Only digits, and dot or coma (locale dependent)
- Only Latin chars
- Only Upper (or only lower) // that may be better with a "set of enum"

Or a String, that contains all allowed chars. If that string is limited to #32..#127 it can be turned into a set for lookup.


Just to say, I know I am going a bit overboard with some of the ideas. Just throwing in everything I can think of.

Especially published properties can be hard to extend later. Enum/set can ease that a bit.

[Martin_fr]

First, a twofold sorry: i'm not sure if it really belongs herein, and: for text search usually i used a separate tool:

No need to be sorry, but yes: wrong thread.

Moved your post to: https://forum.lazarus.freepascal.org/index.php/topic,63601.0.html

This here is not about the search itself.
This here is about "filtering the result list of a search" / I.e. Search in the search results.

[n7800]

Not a button (well maybe a dropdown, see TortoiseGit) ... Maybe make it a context menu, on the filter edit. Well ok, that is for the copy/paste/... stuff, but it could have the rest in there.
See image how it works in TortoiseGit

As you can see in TortoiseGit itself, after each click the menu is hidden and you have to call it again. To search only one field, you need to clear all the others. It will require more than 10 clicks! A lot of time passes... (probably even worth writing about this in the TortoiseGit bug tracker)


And TortoiseGit doesn't allow you to search for something in one "column" and something else in another. As you can see, date and email are selected there in separate components.


In an IDE, you often have to change search terms. And it's more convenient to just type, rather than toggle flags back and forth. It's fast, and only requires a couple of "special" characters.


And then there will not even be such questions:

If it then can be toggled away from the option, then should that
- affect the one filter only (persistent, or for one search)?
- affect all filters

[n7800]

I still think alternatively / optional escaping with \ is a good idea too. Many people are used to that too (like in regex). And if we have global options, then it is easy to toggle.

I don't see any need to reserve one more character. To escape one character (or any number), you can use the same quotes directly in the text:

'text,text' <=> text','text


Incidentally, this character is used as a path separator. I really dislike this in C++:
String path = "C:\\Windows\\System32\\Calc.exe";

[Martin_fr]

All my below quotes were in context of your statement

Code: Pascal  [Select][+][-]

1. the user must enable the multi-word search himself

Not a button (well maybe a dropdown, see TortoiseGit) ... Maybe make it a context menu, on the filter edit. Well ok, that is for the copy/paste/... stuff, but it could have the rest in there.
See image how it works in TortoiseGit

As you can see in TortoiseGit itself, after each click the menu is hidden and you have to call it again. To search only one field, you need to clear all the others. It will require more than 10 clicks! A lot of time passes... (probably even worth writing about this in the TortoiseGit bug tracker)

As indicated, I like the in-search-column selectors.

The tortoise-git was in context of switching the syntax. I.e. current plain search all filter as one text... versus use extended filter syntax.

And then there will not even be such questions:

If it then can be toggled away from the option, then should that
- affect the one filter only (persistent, or for one search)?
- affect all filters

Well that is not about the columns, that is about the search style.

Though I would leave it at one global option "Filters use extended syntax" (checkbox on/off).

[n7800]

Though I would leave it at one global option "Filters use extended syntax" (checkbox on/off).

Well, then I like it. Then you won't need to add an option to each window, everything will be in the filter itself.
You can even do it as a separate button, as there is now a button to clear the filter (if a certain property is set in the filter itself)