Add 'Language' property to Controls with Text?

[Avishai]

Thanks Phil.  I've been trying find out what C# and VB do.  I have read that they have a Language property, but I can't tell if it's only at the Application level, or if it's on the  component level.  I wish I knew.

To me, it's a bit absurd to think that a developer has to consider finding just the right sort method for every possible locale.  That takes a big team that knows a lot of languages.  Lazarus has a big team.

But take a look at Lazarus LowerCase routine.  It's HUGE!  Some one, or a lot of some ones, took a lot of time and trouble to come up with that.  UpperCase is the same way.  Such a very simple task until you realize the complexity of it.

[Phil]

To me, it's a bit absurd to think that a developer has to consider finding just the right sort method for every possible locale.  That takes a big team that knows a lot of languages.  Lazarus has a big team.

I would expect this information to be available from the OS. For examle, on Mac, each locale is associated with a "collation identifier" - that's for localized string comparisons.

Thanks.

-Phil

[Avishai]

That doesn't seem to be the case with Lazarus.  If I use the 'Standard' sort (i.e. TStringList.Sort:= True) on the Hebrew Language, I get a total failure as far as sort order on both English and Hebrew Locales.

Also, having more than one Language in a program isn't that uncommon, so one Sort Order doesn't always work.  Up until recently I was writing only for English and Hebrew, but now I am trying to write for UserLocale + Hebrew and I have to use different sort orders.  It isn't working out too well.

Right now I'm reading MicroSoft's Globalization whitepaper on Sorting (again) trying to make some sense of it.  But knowing the Language is the key.  If that changes from one control to the next, then what?

http://msdn.microsoft.com/en-us/goglobal/bb688122

[Phil]

That doesn't seem to be the case with Lazarus.  If I use the 'Standard' sort (i.e. TStringList.Sort:= True) on the Hebrew Language, I get a total failure as far as sort order on both English and Hebrew Locales.

Design a compare function per your locale (using OS info, if necessary), then pass this into TStringList.CustomSort.

Thanks.

-Phil

[Avishai]

I already do that.  But that's kind of the point of why I'm asking the question.  How can I design a compare function if I don't already know what Locale it will be running on?  I can get the Locale at run-time, but even if I wanted to, I can't write a function for every Language.  And if the app is multi-lingual?

Some time ago I wrote an app for a hotel.  The guests could pick their language by clicking on a flag.  The Locale never changed but the display/entry changed to their language and the correct Keyboard was loaded.

[Phil]

I already do that.  But that's kind of the point of why I'm asking the question.  How can I design a compare function if I don't already know what Locale it will be running on?  I can get the Locale at run-time, but even if I wanted to, I can't write a function for every Language.  And if the app is multi-lingual?

Write the compare function based on information it gets from the OS for the specified locale, whether the system locale or one the app (user) specifies.

Thanks.

-Phil

[Avishai]

That only works if you know which locales you will be working with, or you write one for every locale.  That might be an option on a huge money project, but not for for small projects.  And Locale doesn't address multi-lingual apps.  It gives you only one sort method.

[Phil]

That only works if you know which locales you will be working with, or you write one for every locale.  That might be an option on a huge money project, but not for for small projects.  And Locale doesn't address multi-lingual apps.  It gives you only one sort method.

A "locale" is a language + region designation. You should be able to retrieve information from the OS about any locale you want.

Thanks.

-Phil

[Avishai]

I have no problem getting the information.  But what can I do with it?  There is no Lazarus 'SortJapanese' routine, so you have to know in advance that you need such a sort method.  If you write a program that you want to distribute globally, that would mean having a sort method for every locale.  I have no idea what sort order Finland uses but I am sure that one of the Lazarus users does.  TStringList.CustomSort:= ?

case TStringList.Lang of
  LangEN_US: SortEn_US;
  LangFr: SortFrench;
  LangRu: SortRussian;
  LangHe: SortHebrew;
  ...
end;

If it can be done for LowerCase, it can be done for Sort, but it shouldn't be driven by Locale.  It should be driven by Language.  Right at the moment, my Locale is Hebrew.  But as you can see, I'm working in English.  Imagine if having a Hebrew Locale meant that I could only type Hebrew.  That would mean that I couldn't use Lazarus because FPC doesn't speak Hebrew.  Multi-lingual is a fact of everyday life for most of the world.

[Phil]

It should be driven by Language.  Right at the moment, my Locale is Hebrew.  But as you can see, I'm working in English.  Imagine if having a Hebrew Locale meant that I could only type Hebrew.  That would mean that I couldn't use Lazarus because FPC doesn't speak Hebrew.  Multi-lingual is a fact of everyday life for most of the world.

Option to display a list of locales that your app supports. User chooses one. Load that locale's settings from OS. Etc.

At startup, default to system's locale; once user selects locale, save this in app's preferences and use it in subsequent runs of the app.

Thanks.

-Phil

[Avishai]

I have that option in some of my programs, but Locale doesn't supply a sort order, at least not in Lazarus.  As I said, if I have a TStringList with Hebrew words, I get the same wrong 'Standard' Sort no matter what Locale I use.  That means I have to supply a CustomSort.

( I don't know if the SortLatin is good or not )

function SortHebrew (List: TStringList; Index1, Index2: Integer): Integer;
{ Hebrew }
begin
  Result := StriComp(PChar(List[Index1]), PChar(List[Index2]));
end;

function SortLatin(List: TStringList; Index1, Index2: Integer): Integer;
{UTF8Compare - Portuguese...}
begin
  Result := WideCompareText(UTF8Decode(List[Index1]),UTF8Decode(List[Index2]));
end;

Edit: Also, TListBox and TComboBox don't have a CustomSort.  I have to copy the Strings to a TStringList, Call a CustomSort and copy the strings back to the TListBox or TComboBox.

[Phil]

I have that option in some of my programs, but Locale doesn't supply a sort order, at least not in Lazarus.  As I said, if I have a TStringList with Hebrew words, I get the same wrong 'Standard' Sort no matter what Locale I use.  That means I have to supply a CustomSort.

That's why I'm suggesting that you get your locale info from the OS and that you use a custom sort per whatever your app's requirements are. Add a custom sort as needed for each language as you localize for it.

Remember that TStringList is not part of Lazarus, it's part of FPC RTL.

Thanks.

-Phil

[Avishai]

Wouldn't this make more sense?

Code: [Select]

case TStringList.Lang of
  LangEN_US: SortEN_US;
  LangFR: SortFrench;
  LangRU: SortRussian;
  LangHE: SortHebrew;
  ...
end;

Edit: Wouldn't it be better if you could just say:
 
ListBox1.Lang:= LangRU;
ListBox1.Sorted:= True;

[Phil]

Edit: Wouldn't it be better if you could just say:
 
ListBox1.Lang:= LangRU;
ListBox1.Sorted:= True;

Why not just create your own subclasses of TListBox, etc. and add the properties and functionality you need?

Thanks.

-Phil

[BigChimp]

As far as I understand this problem can be split in 2:
1. non-English sort order is not supported out of the box, period
2. multilanguage forms (i.e. multiple controls with different languages) are not supported
I'd think point 1 is quite bad and should be addressed. Obviously, Avishai feels strongly about 2 - by addressing 1 we might have an easier fix for 2, too.

For problem 1: I agree with Avishai that sorting (whether that is based on locale or language) is something that could and should be provided by the compiler/LCL... that perhaps do a call to OS language-specific sorting functions (Win32 CompareString? as mentioned in Avishai's MSDN link).

Why? As he said, otherwise devs would need to reinvent their own sorting routines for all languages. Perhaps we'd get some forum thread where people share sorting code... fine... but why not add it to the RTL?

I really think it's worth it to look at how other development environments do it - I cannot believe e.g. C#+Windows or Objective C+OSX don't have facilities for this. Perhaps it's a good idea to look at how/if Mono provides this - we could have a look in the source code and have some idea if/how Linux OS calls are used.

Perhaps it's better to just raise a bug on Mantis such as "there's no language or locale dependent sorting in TStringList" with a link to the discussion here to get the devs to weigh in on if/how this could be done.