Lazarus

Miscellaneous => Translations => Topic started by: Raul_ES on April 02, 2020, 02:21:42 pm

Title: My multilingual method for apps
Post by: Raul_ES on April 02, 2020, 02:21:42 pm
Hello,

I've been working in a project for a while and I'm not sure if the multilingual architecture I am currently implementing is really good idea. Sometime ago I had a look to I18n and PO files, but I remember it as a bit confusing and painful to edit, mantein, etc. For some reason I didn't felt confortable using it. Maybe I shall give it a try again someday, but I decided to develop my own method as an alternative and to learn more about pascal. Check it please and let me know what you think.

The method works as follows:

We define a file called constants.inc that contains several constants like LANGUAGES_AVAIBLE = 9 (i.e)
and ES = 1, CA = 2, EU = 3, GA = 4 ... up to the number you need.

The lpr/ppr file will hold a global variable (LANGUAGE : integer) to define a common default language for
all forms in the application:

Code: Pascal  [Select][+][-]
  1. (...)
  2. {$R *.res}
  3. {$INCLUDE ../include/constants.inc}
  4. (...)
  5.  
  6. var
  7.   // crear variables globales
  8.   LANGUAGE : integer; cvar; export;
  9. (...)
  10.  
  11. begin
  12.   LANGUAGE := ES; // default language
  13. (...)
  14.  

Then, for every form in the application we must create an include file (i.e unit_mainform_language.inc)
with the following structure (i.e for eight local languages in Spain and english), that will contain
translations for all the labels, captions, titles, list items, hints, etc, for *that* particular form:

Code: Pascal  [Select][+][-]
  1. {         1              2            3              4           5          6           7           8            9
  2.  +---------------------------------------------------------------------------------------------------------------------+
  3.  |                 |           |              |              |        |           |           |            |           |
  4.  | Español (es_ES) |  Catalán  |  Valenciano  |  Mallorquín  | Aranés |  Euskera  |  Gallego  | Asturiano  | English   |
  5.  |                 |           |              |              |        |           |           |            |           |
  6.  +---------------------------------------------------------------------------------------------------------------------+
  7. }
  8. // the blank ones have not yet been traslated or can be interpreted as equivalent to other's language word (#2 OR #1)
  9.  
  10. uEButton_close_Caption: Array[1..LANGUAGES_AVAIBLE] of String = (
  11. 'Cerrar',
  12. 'Tancar',
  13. '',
  14. '',  
  15. '',
  16. 'Itxi',
  17. 'Pechar',
  18. '',
  19. 'Close');
  20.  
  21.  
  22. Label_copyrightLogo_Caption: Array[1..LANGUAGES_AVAIBLE] of String = (
  23. 'Todos los derechos reservados.',
  24. 'Tots els drets reservats.',
  25. '',
  26. '',
  27. '',
  28. 'Eskubide guztiak erreserbatuta.',
  29. 'Todos os dereitos reservados.',
  30. '',
  31. 'All rights reserved.');
  32. (...)
  33.  

Items in lists, comboboxes and such are declared in two dimensional arrays, something like this:

Code: Pascal  [Select][+][-]
  1. ComboBox_AccessControlType_ListItems: Array[1..LANGUAGES_AVAIBLE,1..7] of String = (
  2. ('Identificador Empleado','PIN','Persona-Clave','Máquina-Clave','Tarjeta','SmartCard','DNI Electrónico'),
  3. ('Identificador Empleat', 'PIN', 'Persona-Clau', 'Màquina-Clau', 'Targeta', 'SmartCard', 'DNI Electrònic'),
  4. ('','','','','','',''),
  5. ('','','','','','',''),
  6. ('','','','','','',''),
  7. ('Langilearen Identifikatzailea', 'PIN', 'Pertsona-Pasahitza', 'Makina-Pasahitza', 'Txartela', 'SmartCard', 'NAN elektronikoa'),
  8. ('Identificador de empregado', 'PIN', 'Contrasinal de persoa', 'Contrasinal de máquina', 'Tarxeta', 'SmartCard', 'Identificación electrónica'),
  9. ('','','','','','',''),
  10. ('Employee Identifier', 'PIN', 'Person-Password', 'Machine-Password', 'Card', 'SmartCard', 'Electronic ID')
  11. );
  12.  


Then in the form's code (i.e unit_whateverform.pas) we must define:

Code: Pascal  [Select][+][-]
  1. (...)
  2. var
  3.   whateverForm: TwhateverForm;
  4.   LANGUAGE : Integer; cvar; external; // it's a global variable
  5. (...)
  6.  
We must place {$INCLUDE ../include/constants.inc} somewere to access the language codes or we may declare those constants as global at the lpr/ppr
file.

Then we will load language translation on creation time to the form:

Code: Pascal  [Select][+][-]
  1. procedure TwhateverForm.FormCreate(Sender: TObject);
  2. var
  3.   //LANGUAGE : Integer; uncommenting this will give an Access violation under linux but not under Windows.
  4.  
  5.   i : integer;
  6.   {$INCLUDE ../languages/unit_whateverform_languages.inc}
  7.  
  8. begin // start to assign translated words
  9.   uEButton_Help.Caption := BitBtn_Help_Caption[LANGUAGE];
  10.   uEButton_Help.Hint := BitBtn_Help_Hint[LANGUAGE];
  11.  
  12.   uEButton_Configuration.Caption := BitBtn_Configuration_Caption[LANGUAGE];
  13.   uEButton_Configuration.Hint := BitBtn_Configuration_Hint[LANGUAGE];
  14.  
  15.   uEButton_Start.Caption := BitBtn_Start_Caption[LANGUAGE];
  16.   uEButton_Start.Hint := BitBtn_Start_Hint[LANGUAGE];
  17.  
  18.   uEButton_Quit.Caption := BitBtn_Quit_Caption[LANGUAGE];
  19.  
  20. (...)
  21.  
  22.  

And we also add items to lists, etc in the selected language:

Code: Pascal  [Select][+][-]
  1. // we then load translated items to lists, comboboxes, etc
  2.  
  3.   for i := 1 to length(ComboBox_AccessControlType_ListItems[LANGUAGE]) do
  4.   begin
  5.      ComboBox_AccessControlType.Items.Add(
  6.                           ComboBox_AccessControlType_ListItems[LANGUAGE][i]);
  7.   end;
  8.   for i := 1 to length(ComboBox_SessionType_ListItems[LANGUAGE]) do
  9.   begin
  10.      ComboBox_SessionType.Items.Add(
  11.                           ComboBox_SessionType_ListItems[LANGUAGE][i]);
  12.   end;
  13. (...)
  14.  

Ok, so that's all. Let me know what you think please.

The main benefits I see this way are:

1) Adding new languages to a *_language.inc file is as easy as using any text editor, copy paste a line "a","b","c"... into Google Translator (if that language is supported) and paste back the result to the inc file at the right position in the array.

2) I believe this can be scripted into an automathed process to add as many languages as supported by Google or other
online services.

3) Easier to distribute plain text files and easier for contributors to edit (I think so)

4) Easier to read and mantein (imho). One file for form with all supported languages. Any one can read the line in english and translate to it's own language.

5) Easier to distribute as inc files are compiled into the executable. No need to distribute and install *.po files

6) You can change language on the fly, creating a loadLanguage() procedure to be called on creation time and at any moment the user wants to. Interesting in a country with more than one oficial language, a POS terminal shared with several operators, etc

7) add your own if you see any...


Bad things I see:

1) words are stored in arrays and data structures that consume memory resources.

2) all languages shall be loaded into memory (probably you dont need to have portuguese, chinese, dutch and arab at the same time, but yes for bilingual regions: finnish-swedish, spanish-catalan, spanish-euskara...)

3) all languages shall be kept in memory while running, at least co-oficial languages, to allow on-the-fly switching. Non required elements can be freed from the array.

4) add what you see here


regards,
Title: Re: My multilingual method for apps
Post by: jwdietrich on April 02, 2020, 02:27:14 pm
I think that this is a good idea. I use a similar, albeit less elegant, approach in my apps.

Dismissing .po files is also beneficial for cross-platform development. .po files are not well supported on certain platforms, e.g. macOS.
Title: Re: My multilingual method for apps
Post by: PascalDragon on April 02, 2020, 02:55:14 pm
Ok, so that's all. Let me know what you think please.

The main benefits I see this way are:

1) Adding new languages to a *_language.inc file is as easy as using any text editor, copy paste a line "a","b","c"... into Google Translator (if that language is supported) and paste back the result to the inc file at the right position in the array.

2) I believe this can be scripted into an automathed process to add as many languages as supported by Google or other
online services.

3) Easier to distribute plain text files and easier for contributors to edit (I think so)

4) Easier to read and mantein (imho). One file for form with all supported languages. Any one can read the line in english and translate to it's own language.

5) Easier to distribute as inc files are compiled into the executable. No need to distribute and install *.po files

6) You can change language on the fly, creating a loadLanguage() procedure to be called on creation time and at any moment the user wants to. Interesting in a country with more than one oficial language, a POS terminal shared with several operators, etc

7) add your own if you see any...


Bad things I see:

1) words are stored in arrays and data structures that consume memory resources.

2) all languages shall be loaded into memory (probably you dont need to have portuguese, chinese, dutch and arab at the same time, but yes for bilingual regions: finnish-swedish, spanish-catalan, spanish-euskara...)

3) all languages shall be kept in memory while running, at least co-oficial languages, to allow on-the-fly switching. Non required elements can be freed from the array.

4) add what you see here

Ehm... you are aware of resourcestring (https://www.freepascal.org/docs-html/current/ref/refse11.html#x23-220002.3)s? (also documented here (https://www.freepascal.org/docs-html/current/prog/progse38.html#x226-2400009.1)) You can use ObjPas.SetResourceStrings and ObjPas.SetUnitResourceStrings to roll your own storage system (as is done with the GetText unit for PO/MO files) while at the same type using the Object Pascal mechanism for this. And by studying LCLTranslator you can also hook yourself into the LCLs translation mechanism (which is also build upon the before mentioned units).
Title: Re: My multilingual method for apps
Post by: jwdietrich on April 02, 2020, 05:13:40 pm
Ehm... you are aware of resourcestring (https://www.freepascal.org/docs-html/current/ref/refse11.html#x23-220002.3)s? (also documented here (https://www.freepascal.org/docs-html/current/prog/progse38.html#x226-2400009.1)) You can use ObjPas.SetResourceStrings and ObjPas.SetUnitResourceStrings to roll your own storage system (as is done with the GetText unit for PO/MO files) while at the same type using the Object Pascal mechanism for this. And by studying LCLTranslator you can also hook yourself into the LCLs translation mechanism (which is also build upon the before mentioned units).

Is there a possibility to create a self-contained multilingual executable with ResourceStrings? I cannot find much information on the possible "own storage system" mentioned by you. Is there a way to implement a mechanism like the one described by @Raul_ES, where the translated strings are stored in an .inc file?
Title: Re: My multilingual method for apps
Post by: kupferstecher on April 02, 2020, 08:31:12 pm
Is there a possibility to create a self-contained multilingual executable with ResourceStrings?
At least with the i18n options and po-files its possible.
https://wiki.freepascal.org/Everything_else_about_translations#Compiling_po_files_into_the_executable_and_change_language_while_running
Title: Re: My multilingual method for apps
Post by: jwdietrich on April 03, 2020, 12:46:22 am
At least with the i18n options and po-files its possible.
https://wiki.freepascal.org/Everything_else_about_translations#Compiling_po_files_into_the_executable_and_change_language_while_running

Thanks, that is very interesting.
Title: Re: My multilingual method for apps
Post by: Roland57 on April 03, 2020, 02:48:29 am
Hello!

@Raul_ES

Interesting example. I didn't know the "cvar, export, external" trick.  :)

I have done something like your example (with little differences) in a project:
https://github.com/rchastain/eschecs/blob/master/source/language.pas

By the way (since you ask for opinions), I don't like translations made with Google. Only a person who is a native speaker of the language and who is an user of the software can do a good translation (IMHO).

Good luck with your project.

 
Title: Re: My multilingual method for apps
Post by: trev on April 03, 2020, 08:10:55 am
By the way (since you ask for opinions), I don't like translations made with Google. Only a person who is a native speaker of the language and who is an user of the software can do a good translation (IMHO).

It's a chicken and egg thing. No X translation, no X users. Google X translation, some X users and then feedback about better X translation :)
Title: Re: My multilingual method for apps
Post by: Roland57 on April 03, 2020, 09:02:17 am
By the way (since you ask for opinions), I don't like translations made with Google. Only a person who is a native speaker of the language and who is an user of the software can do a good translation (IMHO).

It's a chicken and egg thing. No X translation, no X users. Google X translation, some X users and then feedback about better X translation :)

Yes, maybe you are right.

[off topic]
By the way, I never understood this story of chicken and egg. It seems obvious to me that to start a breeding you need a hen and a rooster. What would you do with an egg?  ;)
[/off topic]
Title: Re: My multilingual method for apps
Post by: jwdietrich on April 03, 2020, 05:14:59 pm
[off topic]
By the way, I never understood this story of chicken and egg. It seems obvious to me that to start a breeding you need a hen and a rooster. What would you do with an egg?  ;)
[/off topic]

https://en.wikipedia.org/wiki/Chicken_or_the_egg  ;)
Title: Re: My multilingual method for apps
Post by: jwdietrich on April 03, 2020, 05:27:14 pm
As a first step, it may be helpful to automatically detect the language of the system. If a translation for the current language is available then use it, otherwise use e.g. English.

For one of my projects (SPINA Thyr (http://spina.sourceforge.net)) I developed the following solution:

Code: Pascal  [Select][+][-]
  1. uses
  2.   // ...
  3.   {$IFDEF LCLCarbon}
  4.   , MacOSAll
  5.   {$ELSE}
  6.   {$IFDEF LCLCocoa}
  7.   , MacOSAll, CocoaAll, CocoaUtils
  8.   {$ENDIF}
  9.   {$ENDIF}
  10.   ;
  11.  
  12. function GetOSLanguage: string;
  13.   {platform-independent method to read the language of the user interface}
  14. var
  15.   l, fbl: string;
  16.   {$IFDEF Darwin}
  17.   theLocaleRef: CFLocaleRef;
  18.   locale: CFStringRef;
  19.   buffer: StringPtr;
  20.   bufferSize: CFIndex;
  21.   encoding: CFStringEncoding;
  22.   success: boolean;
  23.   {$ENDIF}
  24. begin
  25.   {$IFDEF Darwin}
  26.   theLocaleRef := CFLocaleCopyCurrent;
  27.   locale := CFLocaleGetIdentifier(theLocaleRef);
  28.   encoding := 0;
  29.   bufferSize := 256;
  30.   buffer := new(StringPtr);
  31.   success := CFStringGetPascalString(locale, buffer, bufferSize, encoding);
  32.   if success then
  33.     l := string(buffer^)
  34.   else
  35.     l := '';
  36.   fbl := Copy(l, 1, 2);
  37.   dispose(buffer);
  38.   {$ELSE}
  39.   {$IFDEF UNIX}
  40.   fbl := Copy(GetEnvironmentVariable('LC_CTYPE'), 1, 2);
  41.     {$ELSE}
  42.   GetLanguageIDs(l, fbl);
  43.     {$ENDIF}
  44.   {$ENDIF}
  45.   Result := fbl;
  46. end;
  47.  

This platform-independent method is able to detect the language as specified by currently logged-in user. With this information it is possible to automatically set the language of the app with the methods suggestd by @Raul_ES, @PascalDragon and @kupferstecher.
Title: Re: My multilingual method for apps
Post by: Raul_ES on April 07, 2020, 04:29:06 pm
Thank you all for your replies  :)
Title: Re: My multilingual method for apps
Post by: Raul_ES on April 07, 2020, 04:36:49 pm
Thanks Roland57

Google may not be the best option, but it's very handy and I have the feeling that their translation algorithms are improving, at least for the languages I know. I think that for isolated words and short sentences actually performs pretty well (sometimes you get a wierd result though). But of course, nothing like a human native speaker to ask for a translation ;-)


regards


Hello!

@Raul_ES

Interesting example. I didn't know the "cvar, export, external" trick.  :)

I have done something like your example (with little differences) in a project:
https://github.com/rchastain/eschecs/blob/master/source/language.pas

By the way (since you ask for opinions), I don't like translations made with Google. Only a person who is a native speaker of the language and who is an user of the software can do a good translation (IMHO).

Good luck with your project.
Title: Re: My multilingual method for apps
Post by: Raul_ES on April 07, 2020, 04:43:00 pm
Quote
For one of my projects (SPINA Thyr (http://spina.sourceforge.net))

Cool!


Title: Re: My multilingual method for apps
Post by: Raul_ES on April 07, 2020, 04:45:15 pm
As a first step, it may be helpful to automatically detect the language of the system. If a translation for the current language is available then use it, otherwise use e.g. English.

...

Thanks, really nice.  I'm going to add it inmediatly to my proyect.

regards
Title: Re: My multilingual method for apps
Post by: winni on April 07, 2020, 05:56:39 pm
Hi!

There is another idea for a multilanguage app.

The idea is "stolen" from the UCSD compiler messages and was built in to the Ventura Publisher in a similar way.

Let the user select on of the languages for which a language file exists.

The language files contain 2 variables in one line:
an integer, a space as separator and a string - like

123 OK
124 cancel
125 continue
126 ....

And in your code you write something like
MyComponent.Caption := Msg (123);

Even with a small Integer you got room for 32.000 entries - so you can leave enough space inbetween for future enhancements.

And you can add (or delete) a language without going into  the code.

Plus: you need only one language file in the memory

Winni
Title: Re: My multilingual method for apps
Post by: Awkward on April 07, 2020, 06:21:28 pm
And in your code you write something like
MyComponent.Caption := Msg (123);

And you ready to remember all your numeric codes or to check your dictionary to find needed text?
Title: Re: My multilingual method for apps
Post by: bytebites on April 07, 2020, 06:29:39 pm
And in your code you write something like
MyComponent.Caption := Msg (123);

And you ready to remember all your numeric codes or to check your dictionary to find needed text?

123
Title: Re: My multilingual method for apps
Post by: winni on April 07, 2020, 07:19:50 pm
Hi!

You just open a language file of a language, that you understand, in the IDE
and with one click you are there.

What helps is to group the messages by numbers, so let's say 100-199 are trvial button captions,
200-299 are questions for user interactions and so on.

Winni
Title: Re: My multilingual method for apps
Post by: Raul_ES on April 16, 2020, 02:41:49 pm
Thanks for your suggestions.

Personally I believe that one language text file for each form containing a grid structure for translated words it's the easier way I've been able to think of. And easier to mantein.


regards
TinyPortal © 2005-2018