BufDataset foCaseInsensitive not working with "ñ" "Ñ"

Forum > Databases

(1/1)

lainz:
"Tempo" is the TEdit text.
The filter works fine for any text, except these special characters.

--- Code: Pascal [+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i < x.length; i++) { x[i].style.maxHeight='none'; x[i].style.height = Math.min(x[i].clientHeight+15,306)+'px'; x[i].style.resize = "vertical";}};} ---Productos.Filtered := False; Productos.FilterOptions := [foCaseInsensitive]; //Productos.Filter := ‘(sNombre = ’ + QuotedStr(‘*’ + Tempo + ‘*’) // + ' OR sCodProducto = ' + QuotedStr(Tempo) // + ' OR sEan = ' + QuotedStr(Tempo) // + ' OR sEan2 = ' + QuotedStr(Tempo)+‘) ’ // + ' AND bActivo <> ' + QuotedStr(‘F’);   Productos.Filter := ArmarFiltroProductos(Tempo); // the same as the commented lines above Productos.Filter := Productos.Filter + ' AND sTipoProducto <> ' + QuotedStr(‘I’); Productos.Filtered := True;
The lowercase of Ñ is ñ, but setting foCaseInsensitive has no effect.

If I type Ñ only uppercase elements of the DB are returned in the DBGrid. Like "Ñandú" but not "Cañón".
If I type ñ only lowercase elements of the DB are returned in the DBGrid. Like "Cañón" but not "Ñandú".

The DB has mixed content, some are upper case and some are lower case.

In BufDataset.pas I found this

--- Code: Pascal [+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i < x.length; i++) { x[i].style.maxHeight='none'; x[i].style.height = Math.min(x[i].clientHeight+15,306)+'px'; x[i].style.resize = "vertical";}};} ---function DBCompareText(subValue, aValue: pointer; size: integer; options: TLocateOptions): LargeInt; begin if [loCaseInsensitive,loPartialKey]=options then Result := AnsiStrLIComp(pchar(subValue),pchar(aValue),length(pchar(subValue))) else if [loPartialKey] = options then Result := AnsiStrLComp(pchar(subValue),pchar(aValue),length(pchar(subValue))) else if [loCaseInsensitive] = options then Result := AnsiCompareText(pchar(subValue),pchar(aValue)) else Result := AnsiCompareStr(pchar(subValue),pchar(aValue));end;
Anyways, the ñ and Ñ are in the ASCII table

http://ascii-table.com/ansi-codes.php

209   D1   U+00D1   Ñ   Latin Capital Letter N With Tilde
241   F1   U+00F1   ñ   Latin Small Letter N With Tilde

Any ideas on how to fix? I'm on Windows but the project works also on Linux and macOS.

lainz:
Attached a simple test project that replicates the Issue, I'm reporting it at the bugtracker too.

How to test:
- Click on the first button, and see the filtered records (it displays only 1 record, must display 2)
- Click on the second button, and see the filtered records (it displays only 1 record, must display 2)

Edit: Reported here
https://bugs.freepascal.org/view.php?id=36512

amartitegui:
Hello
No news on this right?

LacaK:
IMO problem is in AnsiCompareText function. On Windows this functions calls Win API function CompareStringA(), which "Compares two character strings, for a locale specified by identifier" - so SompareStringA(LOCALE_USER_DEFAULT, NORM_IGNORECASE, ...) is called.

And there is remark "If your application is calling the ANSI version of CompareString, the function converts parameters via the default code page of the supplied locale. Thus, an application can never use CompareString to handle UTF-8 text."

But in Lazarus string are by default UTF-8 encoded so probably you supply UTF-8 strings to AnsiCompareText function, which will not work ...

In my experiments on Windows (ansi code page 1250):
- CompareStringA(LOCALE_USER_DEFAULT, NORM_IGNORECASE, 'ábc',3,'ÁBC',3)-2 == 0 ... OK
- CompareStringA(LOCALE_USER_DEFAULT, NORM_IGNORECASE, 'ábc',3,'ABC',3)-2 <> 0 ... OK
- CompareStringA(LOCALE_USER_DEFAULT, NORM_IGNORECASE+LINGUISTIC_IGNOREDIACRITIC, 'ábc',3,'ABC',3)-2 == 0 ... OK

Thaddy:
As per msdn, CompareStrEx should be called for UTF8/16. It looks like it has no impact except for windows versions before Vista.
See the advice here:
https://learn.microsoft.com/en-us/windows/win32/api/winnls/nf-winnls-comparestringa

So this indeed is a bug and needs to be corrected.

Also note that even CompareStrEx has some security considerations as announced as per:
https://learn.microsoft.com/en-us/windows/win32/intl/security-considerations--international-features

Navigation

[0] Message Index