Recent

Author Topic: Add 'Language' property to Controls with Text?  (Read 47443 times)

Avishai

  • Hero Member
  • *****
  • Posts: 1021
Re: Add 'Language' property to Controls with Text?
« Reply #15 on: July 08, 2012, 03:17:45 pm »
OK.  I now have a Sort for TStrings that works for Hebrew. 
I also have a Sort that I was told works for Portuguese. 
It would be nice if some one could check it.  I don't know  Portuguese.
I know that others have Sort routines for other languages and maybe could share. 
My code is not the best, but it's working.

I have tested with TListBox, TComboBox and TMemo. 
I added the Hebrew alphabet in random order into each control and a button to Sort each list.

Lazarus Trunk / fpc 2.6.2 / Win32

Martin_fr

  • Administrator
  • Hero Member
  • *
  • Posts: 5516
    • wiki
Re: Add 'Language' property to Controls with Text?
« Reply #16 on: July 11, 2012, 08:38:56 pm »
I saw your 2 reports on Mantis. While I have no intend to vote for or against it, I do not think that changing the sorting that way makes really sense.

Consider the following words: Add, ball, Can, dot

According to case sensitive, they will be sorted: ball, dot, Add,Can.

That is probably unwanted in the majority of cases. And if they are sorted none case sensitive, then "Add" and "add" are the same word (and a stringlist, if set to reject duplicates) would reject them.

Sorting equal items is normally not possible. They need an additional criteria.
In a Grid, that can be another column. But it could also be something like the original order, or something, that is not available to the component, data stored elsewhere in the app.

----------------
Now having said that, I can of course think of what you might aim for:

   Sort case-insensitive, and only for equal items consider the case.

But that is exactly what I said: It's a case insensitive sort, with use-case specific extra criteria.

Why should the case of identical items be more important than the order in which they were originally added? Or in a grid the content of the next column?

Further more, in a grid, if the word "add" occurs several times, and all of them all lowercase, then they can again not be sorted. But the random result will be seen in the other columns.....

It depends on the application. The generic sort in the LCL can not know what is the most appropriate  sort case.

--------
And further more none of that solves your language (collation) requirements.

This shows, that whatever sort the LCL will provide, it will always fit only a small part of the use-cases.

But if most applications must anyway provide there own sorting, then why should the default be made more complex than needed?

The only way to answer that in a positive manner is, if the new default would match a significantly higher amount of use-cases.
- I do not thing, that the special case-for-equal-items only will do that
- Adding a way to provide a (optional) collation for sorting might help (but I am not sure, it can solve the same-word-diff-case  issue...

------------
Having said all that... I do not know what the best way is.

Something can probably be done. But before going after one small part of it, the overall picture should be considered.

Avishai

  • Hero Member
  • *****
  • Posts: 1021
Re: Add 'Language' property to Controls with Text?
« Reply #17 on: July 11, 2012, 09:20:47 pm »
"Something can probably be done. But before going after one small part of it, the overall picture should be considered."

I totally agree.  Sorting is not an easy task.  Firstly, it is Language Specific.  That is why I started this discussion.  I have some ideas and I am still developing new thoughts.  But I think that Sorting is an important issue for computing.

I think the thing that bothers me most is that it seems that each control has it's own unique sort method, so you get different results for each control if it's not 'English Sort'.  I know that you are far more familiar with Lazarus code that I, but at least on the surface, it would seem like there would be one central Sort method with a decision tree that would be called by all controls that need to Sort.  Even if it's the wrong Sort, at least it's the Same Sort.  But I may be wrong about all of this.  I am still very new to Lazarus/FPC.  I make new discoveries every day :)

I wish I knew more about other languages.  I can only test for English and Hebrew.  That doesn't made a good test.

And then there is the issue about programmaticly activating different Language Keyboards.

Example:  You are in a TMemo. The Hebrew Keyboard is active.  You need to change Font Name in a FontComboBox.  Click on FontComboBox and start to type.  You are still in Hebrew so nothing matches.  Oops. Toggle the Keyboard.  Delete. Select Font.  Back to your memo and start to type... You are not in Hebrew.  Delete.  Toggle Keyboard.
Lazarus Trunk / fpc 2.6.2 / Win32

Avishai

  • Hero Member
  • *****
  • Posts: 1021
Re: Add 'Language' property to Controls with Text?
« Reply #18 on: July 11, 2012, 09:36:13 pm »
Oh By the way.  When TStringList.CaseSensitive:= True you get:

Able,apple,Arrive,cat,City,count...

So it groups 'aA' then 'bB' as I would expect.
Lazarus Trunk / fpc 2.6.2 / Win32

Bart

  • Hero Member
  • *****
  • Posts: 3465
    • Bart en Mariska's Webstek
Re: Add 'Language' property to Controls with Text?
« Reply #19 on: July 12, 2012, 12:24:21 pm »
One might consider introducing a public CompareFunc property for these controls, where user can do something like:

Code: [Select]
function MyCompareFunc(Item1, Item2: Pointer): Integer;
begin
  ..
end;

  ...
  MyListBox.CompareFunc := @MyCompareFunc;
  MyListBox.Sorted :=  True;
  ..

And the sorting algorithm of MyListBox now uses MyCompareFunc() instead of it's default compare function for items.

Bart

Avishai

  • Hero Member
  • *****
  • Posts: 1021
Re: Add 'Language' property to Controls with Text?
« Reply #20 on: July 12, 2012, 04:45:25 pm »
Bart,  that would be a step in the right direction.  It's clumsy the way it is now, but doable.  Adding CompareFunc would help.

Lazarus Trunk / fpc 2.6.2 / Win32

Avishai

  • Hero Member
  • *****
  • Posts: 1021
Re: Add 'Language' property to Controls with Text?
« Reply #21 on: July 12, 2012, 04:55:29 pm »
In LazUTF8.pas there are some routines that are interesting.

function UTF8LowerCase(const AInStr: string; ALanguage: string=''): string;
{ It uses ALanguage to decide how to do LowerCase. }

procedure LazGetLanguageIDs(var Lang, FallbackLang: String);
procedure LazGetShortLanguageID(var Lang: String);
{ Gets the User Locale. }

I haven't really studied at the code to try to see what really happens, but my guess is that at some point, LazGetLanguageIDs gets called and the 'Lang' is used by UTF8LowerCase based on Locale by default.

But you can call UTF8LowerCase and give it any valid 'Lang' if you need to LowerCase a string in another language.

So the idea of having a 'property Lang: String' and 'property ParentLang: Boolean' could be used with these routines to make LowereCase almost automatic.

as for Sorting, something similar could be done.

procedure Sort(AList: TStrings; ALanguage: LongInt);
begin
  case ALanguage of
    LANG_ENGLISH: DoEnglishSort;  // LANG_ENGLISH is in Defines.inc with all the other Langs listed
    LANG_FARSI: DoFarsiSort;         // But LANG_ENGLISH isn't enough.  You need the full LanguageID
    LANG_GERMAN: doGermanSort;
    ...
  end;
end;

I think the hardest part would be finding the right Sort method of each language.  The are A LOT of languages.
« Last Edit: July 13, 2012, 04:31:22 pm by Avishai »
Lazarus Trunk / fpc 2.6.2 / Win32

Avishai

  • Hero Member
  • *****
  • Posts: 1021
Re: Add 'Language' property to Controls with Text?
« Reply #22 on: July 12, 2012, 08:15:15 pm »
Bart, along with your code addition, it looks like it would also need 'property SortAscending: Boolean:= True;' so that you can Sort Ascending or Descending.  So my Hebrew Sort looks like:

function SortHebrew(List: TStringList; Index1, Index2: Integer): Integer;
{Hebrew}
begin
  Result:= UTF8CompareStr(List[Index1],List[Index2]);
  if not SortAscending then
    Result:= -Result;
end;


Of Course all of this can be done with no addition to Lazarus code, but it would make things better (IMO) if it were a part of Lazarus so that not everyone has to re-invent the wheel.
« Last Edit: July 12, 2012, 08:25:25 pm by Avishai »
Lazarus Trunk / fpc 2.6.2 / Win32

Bart

  • Hero Member
  • *****
  • Posts: 3465
    • Bart en Mariska's Webstek
Re: Add 'Language' property to Controls with Text?
« Reply #23 on: July 13, 2012, 12:03:46 pm »
Bart, along with your code addition, it looks like it would also need 'property SortAscending: Boolean:= True;' so that you can Sort Ascending or Descending.

For any TStrings descendant you can use it's CustomSort(@MyCompareFunc), where MyCompareFunc is of type TStringListSortCompare.

But I agree that it would also make sense to have a SortOrder (soAscending, soDescending) property for any class that offers sorting for strings.

It would probably be a major incompatibility with Delphi though.

Bart

BigChimp

  • Hero Member
  • *****
  • Posts: 5740
  • Add to the wiki - it's free ;)
    • FPCUp, PaperTiger scanning and other open source projects
Re: Add 'Language' property to Controls with Text?
« Reply #24 on: July 13, 2012, 12:06:10 pm »
But I agree that it would also make sense to have a SortOrder (soAscending, soDescending) property for any class that offers sorting for strings.

It would probably be a major incompatibility with Delphi though.
@Bart: could you elaborate on that? How does Delphi do sorting for their controls for non-English languages?
Want quicker answers to your questions? Read http://wiki.lazarus.freepascal.org/Lazarus_Faq#What_is_the_correct_way_to_ask_questions_in_the_forum.3F

Open source including papertiger OCR/PDF scanning:
https://bitbucket.org/reiniero

Lazarus trunk+FPC trunk x86, Windows x64 unless otherwise specified

Avishai

  • Hero Member
  • *****
  • Posts: 1021
Re: Add 'Language' property to Controls with Text?
« Reply #25 on: July 13, 2012, 01:22:40 pm »
@BigChimp:  I think because Delphi has been Ansi Chars in the past, it did not need to do anything special.  In Delphi 6, it simply sorts to Char Code.
Lazarus Trunk / fpc 2.6.2 / Win32

BigChimp

  • Hero Member
  • *****
  • Posts: 5740
  • Add to the wiki - it's free ;)
    • FPCUp, PaperTiger scanning and other open source projects
Re: Add 'Language' property to Controls with Text?
« Reply #26 on: July 13, 2012, 01:33:42 pm »
@Avishai: I know Delphi used to use AnsiChar, but it's been Unicode-enabled for a few versions now. I would expect language specific sorting to go with that.

I asked Bart what Delphi was doing because he said a SortAscending property would be incompatible... well, then I suggest we emulate how current Delphi does things (as long as it takes different languages into account and makes sense).
Want quicker answers to your questions? Read http://wiki.lazarus.freepascal.org/Lazarus_Faq#What_is_the_correct_way_to_ask_questions_in_the_forum.3F

Open source including papertiger OCR/PDF scanning:
https://bitbucket.org/reiniero

Lazarus trunk+FPC trunk x86, Windows x64 unless otherwise specified

Bart

  • Hero Member
  • *****
  • Posts: 3465
    • Bart en Mariska's Webstek
Re: Add 'Language' property to Controls with Text?
« Reply #27 on: July 13, 2012, 03:04:04 pm »
@Bart: could you elaborate on that? How does Delphi do sorting for their controls for non-English languages?

My Delphi is very old (3.0), but I suspect Delphi doesn't have this, or else it would have been reported as a missing property/method some tim ago?

Bart

BigChimp

  • Hero Member
  • *****
  • Posts: 5740
  • Add to the wiki - it's free ;)
    • FPCUp, PaperTiger scanning and other open source projects
Re: Add 'Language' property to Controls with Text?
« Reply #28 on: July 13, 2012, 04:16:36 pm »
As far as I understand from the Delphi online help (I don't have a new Delphi either), it should sort by default based on the OS locale, so it should take different sort order into account. Couldn't find a definitive link that clearly indicates how etc.

Confirmation by somebody who actually uses a newer Delphi would be nice though...
Want quicker answers to your questions? Read http://wiki.lazarus.freepascal.org/Lazarus_Faq#What_is_the_correct_way_to_ask_questions_in_the_forum.3F

Open source including papertiger OCR/PDF scanning:
https://bitbucket.org/reiniero

Lazarus trunk+FPC trunk x86, Windows x64 unless otherwise specified

Bart

  • Hero Member
  • *****
  • Posts: 3465
    • Bart en Mariska's Webstek
Re: Add 'Language' property to Controls with Text?
« Reply #29 on: July 14, 2012, 02:21:38 pm »
It probably uses some system library call to compare strings (WideStrings that is), which might be locale dependant 9like StrCmpIW perhaps?).

Bart