Automatic generation of *.pot files for translation

[DirkJ]

When you compile a Lazarus application from within the IDE, a *.pot file containing the text to be translated is automatically generated.
However, if you compile the same application using buildlaz.exe, this *.pot file is not created.

For our automated build process, we need an up-to-date *.pot file. How can we create this file by running a command-line call on a project?

[dbannon]

That really is a good question.
I'd like to see a stand alone executable that you can call to do this, building it into lazbuild would not help if you are doing a direct fpc build.
Davo 

[PeterBB]

How can we create this file by running a command-line call on a project?

Use rstconv
https://www.freepascal.org/docs-html/user/userse45.html

Note; the example should have .pot extension for the output file.
.po files are usually language specific, .pot is the language agnostic template.

[dbannon]

Having read all I can find, I still am unable to understand what rstconv actually does.

There are several stages in the process that ultimately leads to <app>.mo files. Something sweeps through all our source files looking for all the strings expected to be converted, making a .pot file. (Probably using gettext ?) Then Something (else ?) looks through the existing <appname>.<ccode>.po files adding any new ones, deleting, changing, probably using the .pot file.

So, its src->.pot, then .pot->cc.po ? (later of course, translate, then msgfmt)

Lazarus appears to handle the first two steps in its GUI. And does a great job. But if you have source that is not been worked in Lazarus,  or changed after Lazarus is finished with it, how do you achieve the same thing ?

Peter, are you suggesting that rstconv will do the .pot->cc.po conversion. But not sweep through source building the .pot file ?

If rstconv can generate the .pot file from Pascal source it would need all the source files as input files ?  There are (non-pascal) tools to do the pot to cc.po conversions, intelligently, but I have not used them. But sweeping through the pascal source is a pascal problem IMHO.

Davo



 

[PeterBB]

rstconv generates a .pot file from the .rsj file.
The compiler, fpc produces the .rsj file.

When a unit is compiled that contains a resourcestring section, the compiler does 2 things:

    It generates a table that contains the value of the strings as declared in the sources.
    It generates a resource string file that contains the names of all strings, together with their declared values.

https://www.freepascal.org/docs-html/prog/progse39.html#x226-2400009.2

Note also;

If the unit contains no resourcestring section, no [.rsj] file is generated.

[CM630]

One solution, maybe not the best:
1. Open your project
2. Main menu — Project — Project options — Compiler options — Compiler commands — Execute after , paste in the Command: text box the following:

cmd.exe /c "for %f in (lib\$(TargetCPU)-$(TargetOS)\*.rsj) do $(LazarusDir)\tools\updatepofiles.exe %f languages\your_project.pot"

You shall replace your_project with the name of your project.
And obviously it will work this way in windows only, for other OSes you will have to change cmd.exe to some bash or whatever syntax.
And also your_project.pot must exist (as an empty file) before running lazbuild.exe.

[marcov]

Yeah, that is a bad thing. If you have units in some tree (like the LCL) with a .rsj file, they end up in your POT, even if they are not used.

So probably parser the .map to get a list of used units, and use that to filter your RSJ's.

[dbannon]

rstconv generates a .pot file from the .rst file.
The compiler, fpc produces the .rst file.

Or maybe a .rsj file ?  Its a bit confusing, the resource string files might be one of -

• .rsj https://wiki.freepascal.org/Using_resourcestrings the Wiki uses .rsj in an example.
  
• .rsj https://www.freepascal.org/docs-html/prog/progse39.html#x226-2400009.2 prog document 3rd para, near top.
  
• .rst same page as above, but well down in section talking about rstconv.
  
• .rst https://www.freepascal.org/docs-html/user/userse45.html the user manual in an example.
  
  In a simple test on my system it made a .rsj file. However, two more complicated builds refuses to make either.  I am guessing I have an option to FPC set that prevents the creation of resource string files but I sure cannot work out what that is.
  
  

      It generates a resource string file that contains the names of all strings, together with their declared values.
  

  As I noted previously, the resource string files are generated per unit. So, at some stage, they need to be combined into one list with (ideally) no duplicates. And then using rstconv to generate a new .po (or, correctly, .pot) file that can be copied to <app>.cc.po will not make you popular with translators, they expect previous translations to be retained and only new or changed items needing attention.
  
  Lazarus does all of that correctly.
  
  Davo

[CM630]

Maybe it is possible to call lazarus(.exe) with some parameters, instead of lazbuild(.exe).
Gemini said it is possible, but I have not tried.

... will not make you popular with translators, they expect previous translations to be retained and only new or changed items needing attention...

I am not sure that I understood you. As a translator I expect nothing but the latest .PO file. Then in PoEdit I just do Update from POT and that is all.

[PeterBB]

Or maybe a .rsj file ? 
.....
As I noted previously, the resource string files are generated per unit.

Yes its an .rsj file.

There may well be one generated for each unit that has a resource string section.
I avoid the multiple files issue by centralising all resource string definitions in one unit.

[dbannon]

I am not sure that I understood you. As a translator I expect nothing but the latest .PO file. Then in PoEdit I just do Update from POT and that is all.

The .cc.po file is 'maintained'. Its not just a fresh copy of the .pot file every time. The .pot does not contain any translations. So, when the Lazarus PO rebuild happens, new or changed content in the .pot file is merged with the existing cc.po file, leaving existing translations there. Sounds like you use POedit to do that merge rather than Lazarus, thats OK.


@PeterBB

Yes Peter, keeping all translation strings in one file would clearly be an advantage here !   My project has probably half in resourcestr.pas and the remainder all over the place. Sigh ...

But even given that, I am not generating any .rsj files when I do a build. I have two different build models, one using lazbuild the other calls fpc directly and neither make any .rsj files ??  I don't see any option to fpc to determine this, a mystery.

Davo

[PeterBB]

... I am not generating any .rsj files when I do a build. I have two different build models, one using lazbuild the other calls fpc directly and neither make any .rsj files ??  I don't see any option to fpc to determine this, a mystery.

Hi Davo,

I downloaded resoursestr.pas from your tomboy repository,
and tried fpc resoursestr.pas

This is the result;

Code: Pascal  [Select][+][-]

1. fpc resourcestr.pas
2. Free Pascal Compiler version 3.2.2+dfsg-51 [2026/07/06] for x86_64
3. Copyright (c) 1993-2021 by Florian Klaempfl and others
4. Target OS: Linux for x86-64
5. Compiling resourcestr.pas
6. Writing Resource String Table file: resourcestr.rsj
7. 205 lines compiled, 0.1 sec
8.  

Code: Pascal  [Select][+][-]

1. ls -l r*.rsj
2. -rw-rw-r-- 1 peter peter 27569 Aug  9 12:33 resourcestr.rsj
3.  

Cheers,
Peter

[dbannon]

Yep Peter, that is what I was saying, simple cases, like that, work fine. But if you use either lazbuild or a rather complicated script such as buildit.bash, that file does not come into existence exist. Or is, maybe "cleaned up" ?

So, I cannot tell if, for example, rstconv will concatenate the various .rsj files and does it merge the translated .po files ?

I accept lazbuild (and lazarus) do do something much more complicated and therefor skip the creation of .rsj file. However, if, as I do in my second build model (used by Debian),  do it all with FPC I would expect to see that .rsj file (or, maybe those .rsj files).

Anyway, its not really worrying me and DirkJ seems to have left the building.

Davo

[DirkJ]

I’ve been on holiday, which is why I’m a bit late in replying.

I’ve read through the very interesting discussion and tested the various suggestions.

Updating the PO files is part of our translation process. All I need is an up-to-date POT file containing all the latest texts of the application. Ideally, without the LCL texts, as these are located in the /lazarus/languages folder and it’s sufficient to include these.

Regarding our tests:
We created a POT file from *.rsj files in ./lib/$(TargetCPU)-$(TargetOS)/*.rsj using updatepofiles:

Code: Bash  [Select][+][-]

1. touch ./test.pot
2. updatepofiles --searchdir=./lib/x86_64-win64/ ./test.pot

The method using FOR does not work, as you have to pass all files to `updatepofiles` at the same time.

The POT file contains all the texts from the ResourceString sections of all the units used. As expected, the texts from the LCL are missing.

Unfortunately, the texts from the Object Inspector contained in the *.lfm files are missing. These are located in the *.lrj files, which are in the same directory as the *.lfm files.

LazBuild does not create them automatically when building the application.
According to https://wiki.freepascal.org/Lazarus_project_files, you should check these *.lrj files into version control.

Technically, you could also include the *.lrj files in the POT file using updatepofiles.

However, we face the problem that we have forms stored in the standard forms. These are used by several applications, but not every application requires all the forms. This makes it difficult to identify the correct *.lrj files.