Lazarus

Free Pascal => General => Topic started by: dbannon on September 26, 2024, 08:41:08 am

Title: Chinese Fonts in fpPDF
Post by: dbannon on September 26, 2024, 08:41:08 am
I believe that its not possible to display Chinese characters in a PDF using fpPDF (included in the FCL). Further, it seem the font cache in fptty cannot work with truetype fonts that have been 'collected' into a .ttc file. Thats sad as more and more (most?) fonts in Linux and MacOS are .ttc and Windows seems to be catching up too.

So, can interested parties have a look at my 'simple' demos and see if I have missed something, I honestly know very little about Chinese fonts !  For the record, I don't think its a UTF8 3 byte issue, other font sets using 3 bytes seem to work OK. The TTF fonts I use do seem, themselves happy to display Chinese characters on screen, just not in a the PDF.

I have attached two demo source files, both derived partly from the examples provided with the fpPDF unit. One is, perhaps the simplest code to make a PDF, down side is it requires hardwired paths to font files, it does not use the gTTFontCache that can make that easy.

The second demo source file, uses the cache, is capable of finding the fonts you request all by itself. But a bit harder to understand with a quick (or even slow) glance.

Both files are purely command line driven and are compiled with just a simple fpc line. I suggest at least FPC-Fixes, quite a lot of work seems to have gone into the caches, particularly for Windows since 3.2.2 (thanks!).

I am working with FPC-Fixes on Linux (Debian 12) and Windows 11.

I have included only the FreeSans font in the example zip because thee specialist Chinese ones are too big.

David
Title: Re: Chinese Fonts in fpPDF
Post by: Thaddy on September 26, 2024, 02:15:28 pm
I have included only the FreeSans font in the example zip because thee specialist Chinese ones are too big.
From experience I know that it can bite you if you use the wrong font. In fppdf it is assumed you use a correct font, although there might be some issue with font families where it is possible that the wrong font substitution is made.
My font of choice is NSimSun which should work with fppdf. (I use that font for all my Chinese language related answers.)
Let me know if that works. And yes, it is big..
Note it not only covers simplified Chinese but also Japanese and Cyrillic and latin alphabets.
https://learn.microsoft.com/en-us/typography/font-list/simsun
This font can also be used on Linux.
Note that my knowledge of Chinese is very, very limited, but not absent.
Title: Re: Chinese Fonts in fpPDF
Post by: rvk on September 26, 2024, 05:07:32 pm
The problem might be that SimSun-ExtB is a .ttf file and SimSun/NSimSun is a .ttc file.
See https://learn.microsoft.com/nl-nl/typography/font-list/simsun

And in fpTTF.pp only .ttf and .otf files are read when building the FontCache.

Code: Pascal  [Select][+][-]
  1.         if (lowercase(ExtractFileExt(s)) = '.ttf') or
  2.            (lowercase(ExtractFileExt(s)) = '.otf') then
  3.         begin

So... .ttc files are not read into the FontCache and you can't use them with Doc.AddFont.

Of course you could try
Code: Pascal  [Select][+][-]
  1.     FontCh := Doc.AddFont('c:\windows\fonts\simsun.ttc', 'NSimSun');
but my guess is that fpPDF doesn't handle .ttc files at all !!

From TPDFDocument.LoadFont()
Code: Pascal  [Select][+][-]
  1.   if FileExists(lFName) then
  2.   begin
  3.     s := LowerCase(ExtractFileExt(lFName));
  4.     Result := (s = '.ttf') or (s = '.otf');
  5.   end
  6.   else
  7.     Raise EPDF.CreateFmt(rsErrReportFontFileMissing, [lFName]);

My font of choice is NSimSun which should work with fppdf. (I use that font for all my Chinese language related answers.)
Let me know if that works. And yes, it is big..
So Thaddy, can you show us a short simple example where you use NSimSun in fpPDF ??


(BTW. I think mORMot2 does have support for SimSun and ttc fonts)
Title: Re: Chinese Fonts in fpPDF
Post by: dbannon on September 27, 2024, 02:58:55 am
Thanks Thaddy, RVK.

The problem might be that SimSun-ExtB is a .ttf file and SimSun/NSimSun is a .ttc file.

Indeed rvk, the font cache (fpttf) will not load .ttc file.  It does not understand the format. And fpPDF will not load .ttc files either. So, .ttc files are ruled out.

The font cache does load .otf fonts and seems to understand them but when it it passes the font file name to fpPDF, it fails. Actually seems to try to load the font but crashes when the document is being finalised. Same thing happens if you supply the font file directly into fpPDF.

Thats why I included a demo that does NOT use the fontcache. The font name and path is "hard wired" in the call to Doc.AddFont().

One problem is fpPDF can work with ONLY .ttf files. Another is that even with a .ttf file (that does have Chinese characters) it does not display them.

The issues with the font cache are separate and, perhaps, incidental, it does not work with .ttc files. They are forecast to be future of fonts being space effective and I guess far better on a platform like Windows with poor disk i/o performance.

I read in one place that .ttf fonts differentiate between "screen fonts" and "printer fonts" and .otf fonts do not have that problem. But I could not find any further details of this claim. But as fpPDF is not interested in .otf fonts anyway ....

Quote
So Thaddy, can you show us a short simple example where you use NSimSun in fpPDF ??
Indeed, that would be useful. My Windows install does not have NSimSun but it would be trivial, on a system with it, to modify the simpleFPpdf.pas demo I provided to test it.

Quote
(BTW. I think mORMot2 does have support for SimSun and ttc fonts)

Even my app, tomboy-ng, will display Chinese characters on screen and printed. Just using the default Desktop font, Cantrell and, just tested, using Nimbus Sans, a .ttc font. (Linux, Debian 12).  No special action taken in either screen display of printing, just request that font.

Thanks for your interest.

Davo
Title: Re: Chinese Fonts in fpPDF
Post by: Thaddy on September 27, 2024, 07:21:16 am
You could also try the noto sans font.
The ttf version of simsun is simsunb.ttf (huge)
You can use a tool like fontforge to extract ttf's from ttc files. The version in the ttc file is smaller, but still big.
If the font is ttf it will work with fppdf.
I have no example available, but it should always work.
Title: Re: Chinese Fonts in fpPDF
Post by: dbannon on September 27, 2024, 08:41:00 am
You could also try the noto sans font.
I tried using noto sans CJK SC, its part of a .ttc collection and does not work. So, I tried using fontforge to extract (actually reconstruct) the ,ttf file and, also, that did not work.
Quote from: Thaddy
The ttf version of simsun is simsunb.ttf (huge)
I understand that is a Windows (propriety font). Yes, of course its huge, there are something like 90,000 pictographs in the base character set. Do a bold, italics version of that and, wow !  A couple of days ago I did try getting the .ttf file from some open source CJK font collections, just as big to start with, Came down to a relatively small .ttf file but made no difference.

Quote from: Thaddy
If the font is ttf it will work with fppdf.

Sigh...
Thaddy, please read my long winded story above. I have tried all these approaches several days ago. It did not work. I extracted the "Noto Sans CJK SC" .ttf file and used it directly, without the font cache. It did not work. I tried simsun-extB on Windows and it did not work. I have other fonts here that include Chinese characters, they display on screen and print to my printer from my app. But they do not produce the goods when used with fpPDF.
Quote from: Thaddy

I have no example available, but it should always work.
Thaddy, my first post in this thread contains a zip file, two source files with .pas extension. The one called simpleFPpdf.pas is quite short, around line #71, replace my reference to "fonts/simsunb.ttf" with a full or relative reference to the font you believe does work and please try it. If it cannot be made work with like that, either there is something wrong with my code (what?) or it just does not work.
 
Davo
Title: Re: Chinese Fonts in fpPDF
Post by: paweld on September 27, 2024, 01:30:18 pm
Attached is a working example - I checked on Windows 10 and Debian 12.
In the directory with the program, place the files (you only need one) TTF: NotoSerifCJKsc-VF.ttf [ https://files.brudnopis.ovh/file/SlUi3ZVso1en7Lc4/aWCITfFAib6AODWI/NotoSerifCJKsc-VF.7z ~22MB] and NotoSansCJKsc-VF.ttf [ https://files.brudnopis.ovh/file/SlUi3ZVso1en7Lc4/R0OEY1sXE5j1hjBG/NotoSansCJKsc-VF.7z ~13MB] (or the whole packages can be downloaded from the author's repository: https://github.com/notofonts/noto-cjk/releases )

I am working in Lazarus trunk with FPC 3.2-fixes, but I checked and it works with Lazarus 3.2 with FPC 3.2.2 without any problem
Title: Re: Chinese Fonts in fpPDF
Post by: rvk on September 27, 2024, 02:48:16 pm
I also think it is possible to convert the simsum.ttc to simsum.ttf. and use that.
You probably need to embed the font (of subset) in that case.
Title: Re: Chinese Fonts in fpPDF
Post by: Thaddy on September 27, 2024, 06:01:58 pm
Note that the ttc format is quite easy too: it is just a header and a directory to a series of ttf fonts.
Title: Re: Chinese Fonts in fpPDF
Post by: rvk on September 27, 2024, 06:11:04 pm
Yeah, I see it. Just a SimSun-01.ttf and NSimSun-02.ttf (last one should be monospaced).

fpPDF could have easily done that  ;)
Title: Re: Chinese Fonts in fpPDF
Post by: dsiders on September 27, 2024, 09:44:18 pm
Yeah, I see it. Just a SimSun-01.ttf and NSimSun-02.ttf (last one should be monospaced).

fpPDF could have easily done that  ;)

I did not even know about .ttc collections. Submit a feature request...
Title: Re: Chinese Fonts in fpPDF
Post by: dbannon on September 28, 2024, 02:41:47 am
Attached is a working example -
....

Wow, yep, you example works for me too. The difference between your code and mine is poSubsetFont as an option to the (pdf) doc. That appears to be incompatible with the Chinese fonts (but OK with all the others). Turning it off does not prevent font embedding (of a .ttf font) so, no longer trying to use .ttc files, I'll leave it off and do some more testing.

I am somewhat confronted by font files compressed with 7z, thats a new one on me !  But seems to work.  And I will need to experiment with the .ttf files I extracted using FontForge from Noto Sans CJK SC (rather than from your web link). I expect mine will work too now I have poSubsetFont turned off. Sigh...

Thanks heaps paweld, for restoring my faith in fcl-pdf, I was convinced it was a far more serious problem !

Davo

edit: clarify need for poSubsetFont

Title: Re: Chinese Fonts in fpPDF
Post by: jianwt on September 28, 2024, 02:52:15 am
Hope to have a perfect solution for Chinese, I use mORMot2 to generate PDF files, encounter Chinese font is also reported error!
Title: Re: Chinese Fonts in fpPDF
Post by: dbannon on September 28, 2024, 03:14:55 am
I also think it is possible to convert the simsum.ttc to simsum.ttf. and use that.
You probably need to embed the font (of subset) in that case.

Thats why I had poSubsetFont turned on in my example. And that was the cause of my problems.

simsum is a Microsoft proprietary font, so, the license may no permit such deconstruction.

Further, and I am on seriously shakey ground here, I suspect the .ttc fonts I was looking at are sometimes collections of .otf fonts rather than .ttf . Thats based on superficial observations of what FontForge was showing me. To make it more difficult, apparently its not unusual to see the extensions .otf and .ttf used interchangeably, nearly so but not correct.

And I still believe that .otf fonts don't work in fppdf (but wonder if that was also a poSubsetFont issue ?). More research is indicated.

Davo
Title: Re: Chinese Fonts in fpPDF
Post by: dbannon on September 28, 2024, 08:23:49 am
OK, here is a summary of what I have found. Firstly, just about all of what I have said further up this document is either wrong or dodgy !  And for that I apologize !

The reason why paweld's code worked (and mine did not) is primarily because of the fppdf doc option poSubsetFont, setting it totally messes with Chinese .ttf fonts and .otf fonts. (I suspect that some chinese .tty fonts are, in fact .otf.). Further rambling notes -



My model in my application will be to, offer a window that allows user to specify a font they would rather use in the PDF. If they do so, I will check that the font can be found (by the Cache) and, if so, that it is a .ttf or .otf.

As an alternative, I could look at each character, one by one, testing first to see if its a three byte character, then that its in the CJK range, then the above test to see if we can make it usable. This approach would then be able to handle extended European and CJK in the same document. But would be substantially slower and require more code. PDF is quite incidental to my app.

Thanks folks for the very positive help you have provided here, especially paweld who produced that magic code !

I will get a lot of this information on to the wiki as it becomes more proven. Will post here when done.

Davo





Title: Re: Chinese Fonts in fpPDF
Post by: paweld on September 28, 2024, 09:06:56 am
@Davo
- I made the fonts available in a 7z archive because they take up much less space, which also helps with a poor internet connection
- if you disable built-in fonts in the options, the poSubsetFont option disables automatically, but then you need to have all the fonts installed for the pdf to display properly
Title: Re: Chinese Fonts in fpPDF
Post by: dbannon on September 28, 2024, 11:45:29 am
Right, attached is my demo based on what I have learnt here.

It builds and runs on both Linux and Windows (and probably MacOS). Just needs fpc to build from the command line, runs from the command line. If you have an appropriate font installed or in the same dir (or below) as the executable it does show the Chinese characters.

However, neither my 'out of the box' Linux or Windows have appropriate fonts installed. You need a Chinese specialist font, either a .ttf or .otf file. Not a .ttc file. I suggest using a .ttf font from https://github.com/notofonts/noto-cjk

I'd expect Chinese speakers will have more choice and it would be good to hear if that is the case.

paweld - yes, I realised how silly I was being wrt the 7z files, once I realised I was not using them !

Yes, its necessary to embed fonts IMHO. Only exception is if you are using the so called 14 PDF fonts and they are really not suitable. fpPDF default is to embed fonts, you turn it off by asserting poNoEmbeddedFonts. The default for poSubsetFont is off but the examples I found were on. I should have looked closer. Its not totally clear to me what it does, but it sure breaks .otf fonts and, I suspect, .otf fonts masquerading as .ttf.

I am pretty happy with what we have now !  Some notes on the wiki will be a good idea !

Davo

Title: Re: Chinese Fonts in fpPDF
Post by: jianwt on September 29, 2024, 03:31:03 am
Unfortunately, I put a Chinese font "simfang.ttf" in the fonts directory, which is the "中文仿宋" font, and put it in the "FontsCh: TStringArray = ('simfang','Noto Sans CJK SC', 'Noto Serif CJK SC', 'Source Han Serif SC VF'{, 'Simsun-ExtB'}); " The name is added, but when the PDF file is generated, the Chinese is still not displayed, only the box is displayed.The program prompts that the Chinese font cannot be found.
Title: Re: Chinese Fonts in fpPDF
Post by: paweld on September 29, 2024, 06:11:27 am
@jianwt Because you have to specify the font name instead of the file name. For “simfang.ttf” it is “FangSong” (case sensitive).

To check the font name you can modify the code attached by @dbannon as follows:
Code: Pascal  [Select][+][-]
  1. function TPDFTestApp.SetupFonts(Doc: TPDFDocument): boolean;
  2.   var
  3.     i: integer;
  4.   begin
  5.     //...
  6.     for i := 0 to gTTFontCache.Count - 1 do
  7.     begin
  8.       if pos('CJK', gTTFontCache.Items[i].HumanFriendlyName) > 0 then
  9.         writeln('FONT-' + gTTFontCache.Items[i].FileName + ' HF=' + gTTFontCache.Items[i].HumanFriendlyName)
  10.       else if pos('simfang.ttf', gTTFontCache.Items[i].FileName) > 0 then
  11.         WriteLn('!!! for file "simfang.ttf" font name is: ', gTTFontCache.Items[i].FamilyName);
  12.     end;
  13.     //...
  14.   end;    
And then after defining the list of Chinese fonts to:
Code: Pascal  [Select][+][-]
  1. FontsCh: TStringArray = ('FangSong', 'Noto Sans CJK SC', 'Noto Serif CJK SC', 'Source Han Serif SC VF'{, 'Simsun-ExtB'});  
we get the result as below
Title: Re: Chinese Fonts in fpPDF
Post by: jianwt on September 29, 2024, 07:21:40 am
@paweld 

I did. It's just a mistake. Can you send up your project? Thank you very much!
Title: Re: Chinese Fonts in fpPDF
Post by: paweld on September 29, 2024, 09:16:40 am
this is a reworked design by @dbannon
I downloaded the font from: https://github.com/21wmd/ttf-simfang/blob/master/usr/share/fonts/truetype/sim/simfang.ttf and uploaded it to the project directory
Title: Re: Chinese Fonts in fpPDF
Post by: dbannon on September 29, 2024, 12:11:27 pm
this is a reworked design ...

Yep, definitely works fine for me.  A pain searching for FileName reather than Human Friendly Name but only way to find whats needed. 

jianwt, unless you have a reason for using FangSong, you might be better using "Noto Sans CJK SC". It, and its friends are pretty well documented and available in a range of 'weights', that is Light..Black, Mono spaced, with and without Serif.

https://github.com/notofonts/noto-cjk

https://github.com/notofonts/noto-cjk/tree/main/Serif#downloading-noto-serif-cjk

Davo
Title: Re: Chinese Fonts in fpPDF
Post by: jianwt on September 29, 2024, 03:05:35 pm
@paweld。

“FontsCh:  TStringArray = ('FangSong', 'Noto Sans CJK SC', 'Noto Serif CJK SC', 'Source Han Serif SC VF'{, 'Simsun-ExtB'}); ” From what did you get the name 'FangSong'? Is it Chinese pinyin?

Why can't I just change it to “FontsCh:  TStringArray = ('仿宋', 'Noto Sans CJK SC', 'Noto Serif CJK SC', 'Source Han Serif SC VF'{, 'Simsun-ExtB'}); ” in Chinese.
Thanks to paweld AND dbannon  for their help.
Title: Re: Chinese Fonts in fpPDF
Post by: dbannon on September 30, 2024, 01:34:44 am
Hi jianwt,

The word "FangSong" is hard wired into the font, maybe "仿宋" is too, I don't know.

But those fonts are read and indexed by the Font Cache, in the fpttf unit and its very likely that it looks, when trying to find what you want, at only the "English" name. That could be because all fonts have an English name, maybe its quicker and more reliable to search for that English name ?  I really don't know.

It could be something like the specification for TTF fonts defines the name of the font as having only Latin characters, that is the 7 bit English alphabet. I have never seen a font, for example that uses even a European accented character in its name. Again, I don't know. The specification for TTF must be pretty old now....

Davo
Title: Re: Chinese Fonts in fpPDF
Post by: jianwt on September 30, 2024, 03:23:13 am
@paweld AND @ dbannon

I use more Chinese fonts, such as "simfang.ttf" Chinese font "FontsCh" = "fangsong", this name, with what method or software can be found? "FontsCh: TStringArray = ('fangsong', 'Noto Sans CJK SC', 'Noto Serif CJK SC', 'Source Han Serif SC VF'{, 'simsun-ExtB'});" . I'm only guessing that 'fangsong' is the Chinese pinyin of the font.
Title: Re: Chinese Fonts in fpPDF
Post by: jianwt on September 30, 2024, 03:46:58 am
Using the font viewer, it turns out that's the English name of the font. We can find out. Got it. Thank you.
Title: Re: Chinese Fonts in fpPDF
Post by: jianwt on September 30, 2024, 05:04:23 am
@ dbannon
In lazarUS3.4, linux 64 system under the compilation report overflow error "项目testfppdf提引异常类"EListError'带有信息:Listindex (-1) outof bounds  在地址431C69", the code did not change anything. Please take a look.
Title: Re: Chinese Fonts in fpPDF
Post by: dbannon on September 30, 2024, 09:15:12 am
Sorry, cannot uncompress a .rar file. Could you send a zip or tgz file please ?

Davo
Title: Re: Chinese Fonts in fpPDF
Post by: 440bx on September 30, 2024, 09:30:02 am
just FYI, 7zip can decompress most .rar files. 
Title: Re: Chinese Fonts in fpPDF
Post by: jianwt on September 30, 2024, 10:07:48 am
@ dbannon
zip
Title: Re: Chinese Fonts in fpPDF
Post by: dbannon on September 30, 2024, 11:02:38 am
OK jianwt, I see the problem.  Down in the function TPDFTestApp.SetupFonts() you have this code starting at line number 226 -

Code: Pascal  [Select][+][-]
  1.     {$ifdef windows}
  2.     gTTFontCache.SearchPath.Add('./');
  3.     // Put 'extra' fonts right next to binary ??
  4.     gTTFontCache.ReadStandardFonts;
  5.     // Scans usual places but apparently also ./fonts/ ? Nice ...
  6.     {$endif}        

That means the "gTTFontCache.ReadStandardFonts;" command is only used in Windows, not linux.  Having the fonts next to binary is a good idea in Windows, a very bad idea in Linux. But both MUST call gTTFontCache.ReadStandardFonts;. So, this is better -

Code: Pascal  [Select][+][-]
  1.     {$ifdef windows}
  2.     gTTFontCache.SearchPath.Add('./');
  3.     // Put 'extra' fonts right next to binary ??
  4.     {$endif}
  5.     gTTFontCache.ReadStandardFonts;
  6.     // Scans usual places but apparently also ./fonts/ ? Nice ...
  7.        

Why ?  Generally, a Windows install will have the application in its own folder so, searching there is quick. But installed binaries will probably go into /usr/bin/ and I have 2649 files there and many in lower levels so, you don't want gTTFFontCache scanning that !

I tried it with that change, all good.

I have my user supplied fonts in $HOME/.fonts - thats a great place on Linux. At least for now, I note in the libfontconfig source, a note that they will stop looking some time ??  But will still work well with gTTFFontCache

Good luck

Davo

Title: Re: Chinese Fonts in fpPDF
Post by: jianwt on October 01, 2024, 05:58:08 am
@ dbannon

Because I just want to generate PDF files under linux system. I put the Chinese font "SIMFANG.TTF" I used under './', '/fonts' and' /home/username/.fonts' and couldn't find them. Please help me have a look again, I will upload the fonts, and please help me correct where I need to modify my code.

Due to the use of Chinese font cannot upload with package, I give a download address  https://github.com/21wmd/ttf-simfang/blob/master/usr/share/fonts/truetype/sim/simfang.ttf
Excuse me.
Playing with code for half a day, the program is running when the overflow. Can you help me see where my code is wrong and how to fix it?
Thank you very much。

Title: Re: Chinese Fonts in fpPDF
Post by: dbannon on October 01, 2024, 06:42:53 am
jianwt, I just compiled and run you recent example code, it worked, for me, as expected.

I believe your problem must relate to finding the font. In my case, I have simfang.ttf in my ~/.fonts directory and the FontCache does look there, does find it. We need to understand why it does not find it i your case.

When you run the compiled app, you should see a report of what it did. If running from Lazarus, you need to view that report form Consol I/O but I suggest it would be much, much easier if you ran it from the command line. So, please do the following -
Compile the code (from withing Lazarus if you like)
Then, open a terminal, and change to the directory where the source code is.
Then give the following command, one by one -
Code: Bash  [Select][+][-]
  1. $> ls
  2. $> ./testfppdf

Then copy that output from those commands by highlighting them with the mouse and pressing Ctrl-Shift-C.  Now, go back to your browser and paste that content into a response to this thread.

Post that message.

Then, back to your terminal and give this command -

Code: Bash  [Select][+][-]
  1. $>  ls  -la  ~/.fonts

And copy the output from that into another new message and post it too. That should be all we need to do to work out what is happening. I am sure we will work it out !

Davo
 

Title: Re: Chinese Fonts in fpPDF
Post by: dsiders on October 01, 2024, 06:58:33 am
To my knowledge, you need make sure the new font(s) are in the font cache.

Code: Bash  [Select][+][-]
  1. fc-cache -f -v
  2. fc-list
  3.  
Title: Re: Chinese Fonts in fpPDF
Post by: jianwt on October 01, 2024, 08:32:40 am
When the operation goes to "Then copy that output from those commands by highlighting them with the mouse and pressing Ctrl-Shift-C. Now, go back to your browser and paste that content into a response to this thread. "During this step, Exception at 0000000000431BF9 is displayed: EListError: List index (0) out of bounds. ". I'm a novice, I don't know how to operate, just copy the error message. Still don't know what's causing the problem; I looked at it in detail, and the Chinese font of the letter I placed should have been found, but it still said "Exception at 0000000000431BF9: EListError: List index (0) out of bounds." I don't know what to do. Beginners are terrible. Thank you very much. I'm really sorry for taking up so much of your time.

It may also have something to do with the deepin64 system I use.

jianwt@jianwt-PC:~/Desktop/testFPpdf_linux$ ls  -la  ~/.fonts
总用量 32112
drwxr-xr-x  2 jianwt jianwt     4096 10月  1 10:04 .
drwxr-x--- 25 jianwt jianwt     4096 10月  1 14:18 ..
-rw-r--r--  1 jianwt jianwt  3996872 6月  14 11:26 仿宋_GB2312.ttf
-rw-r--r--  1 jianwt jianwt 12128500 6月  14 11:26 华文中宋.ttf
-rw-r--r--  1 jianwt jianwt  4135804 6月  14 11:26 楷体_GB2312.ttf
-rwxrw-rw-  1 jianwt jianwt   980756 12月  7  2019 ARIALBD.TTF
-rwxrw-rw-  1 jianwt jianwt  1036584 12月  7  2019 ARIAL.TTF
-rwxrw-rw-  1 jianwt jianwt 10578152 10月 15  2019 SIMFANG.TTF
-rw-r--r--  1 jianwt jianwt       36 6月  14 11:26 .uuid
Title: Re: Chinese Fonts in fpPDF
Post by: dbannon on October 01, 2024, 09:24:46 am
To my knowledge, you need make sure the new font(s) are in the font cache.

Don, you certainly do if you use libfontconfig directly but when running the Cache from fpttf it seems to read from ~/.fonts as long as you call .ReadSdFonts.  Or so I have found.

Davo
Title: Re: Chinese Fonts in fpPDF
Post by: jianwt on October 01, 2024, 09:58:38 am
I also put the fonts under "/usr/share/fonts/" and updated the font cache. But still reported "During this step, Exception at 0000000000431BF9 is displayed: EListError: List index (0) out of bounds.", very helpless! .
Title: Re: Chinese Fonts in fpPDF
Post by: dbannon on October 01, 2024, 11:47:25 am
jianwt, I believe that you are getting this exception because you are trying to use a font that does not exist. But I am sure that the code you are using checks for such a situation !

Attached is the code for a much simpler test, it tests only the cache, you can run it to prove, or otherwise, that you have the fonts available that your expect.

Please compile it in Lazarus and then, from the command line in the directly it was compiled in, run, for example -

$> ./project1  --file=simfang.ttf

That should show us, conclusively, that the font is being found in ~/.fonts or NOT !

I note you have changed the file, simfang.ttf to SIMFANG.TTF, strange thing to do but my tests indicate that fpttf is case insensitive.

Please run my little test, I am pretty sure it cannot do anything that will raise an exception.

(except, if you get the parameter wrong, don't think I checked for that ...)

Davo

Title: Re: Chinese Fonts in fpPDF
Post by: jianwt on October 02, 2024, 09:47:58 am
The forum can not log in for a day, just can log in.
Sometimes the forums don't go up.

After compiling, the system displays
 
【jianwt@jianwt-PC:~/Desktop/TestCache$./project1 --file=simfang.ttf

Found 245 fonts.】

I think I found the font. Does' 245 'represent a single font, or the sum total of fonts in my system? Don't understand??

I fill in SIMFANG.TTF because when I paste the font into the linux system, it becomes uppercase.

I want to start from simple, you can help me write a Chinese font DEMO, simple point, see what the problem is.
Title: Re: Chinese Fonts in fpPDF
Post by: paweld on October 02, 2024, 01:28:00 pm
Attached an example requires “pdftoppm” installed
Title: Re: Chinese Fonts in fpPDF
Post by: dbannon on October 02, 2024, 01:56:03 pm
OK, jianwt, I am sorry, I made a mistake with that code.  Sorry, I was in a hurry and I did not test it properly !!!

You could repeat that test with a different command, please try -

Code: Bash  [Select][+][-]
  1. $> ./project1 --part=fang

Attached is a second version that works as it should. You can recompile with is code and try again. But the above test should work.  The test command takes three options -



Please either try again, with the existing copy and the new command I mention above.  OR, recompile and try the original command.

Again, sorry for my mistake.

The 245 fonts is how many fonts were found on your system by the cache.  You have less than I would expect, my standard Debian install has something like 2900. So, that says to me, maybe you don't have the default fonts we are assuming are present ?

> I fill in SIMFANG.TTF because when I paste the font into the linux system, it becomes uppercase.

Wow, I have no idea why that would happen. But my tests seemed to indicate the FontCache is case insensitive. If the above tests don't show us anything interesting, just as another thing to try, please do this -

Code: Pascal  [Select][+][-]
  1. $>  cp  ~/.fonts/SIMFANG.TTF  ~/.fonts/simfang.ttf

And then run the above experiments.  I don't think this will make any difference but please try, just in case.

And another experiment, if you are still not getting an answer from projet1, try this -

Code: Pascal  [Select][+][-]
  1. $> sudo  cp ~/.fonts/simfang.ttf /usr/local/share/fonts/truetype/.

You will have to enter the root password when prompted.  Please don't do this if you have had a positive response to the experiments above here.

Davo

ps, when I run that command, it looks like this -

Code: Bash  [Select][+][-]
  1. dbannon@dell:~/Pascal/fcl-pdf/TestCache$ ./project1 --part=fang
  2. Found 2948 fonts.
  3.         Name - [FangSong]
  4.         File - [/home/dbannon/.fonts/simfang.ttf]
  5.         Regular

EDIT: forgot the attachment !
Title: Re: Chinese Fonts in fpPDF
Post by: dbannon on October 02, 2024, 02:30:25 pm
jianwt, in case my theory is right, that you don't have any of what we consider "standard fonts", here is a stripped down version of the earlier PDF demo, it uses ONLY simfang (or, failing that, one of the other Chinese fonts we know work).  It does NOT use things like liberation sans ....

If it does not find a suitable chinese font, it will give an error message, it "should" not crash. If it does, then we need to work out why !

Only prints one block of text using a chinese font.

Davo

Title: Re: Chinese Fonts in fpPDF
Post by: jianwt on October 02, 2024, 04:22:37 pm
@paweld

@dbannon

Thank you. I'll take a good look at your routine in the morning.
Title: Re: Chinese Fonts in fpPDF
Post by: jianwt on October 03, 2024, 03:46:49 am
@paweld

Thank you for your help, I have run your DEMO and it runs perfectly in my linux system. Thanks again for your help! Wish you a happy work!
Where can I get the file "pdftoppm"? Can you give me a download website? thank you.
The operation effect is shown in the following figure:
Title: Re: Chinese Fonts in fpPDF
Post by: jianwt on October 03, 2024, 03:54:05 am
@dbannon

Your last TestPDF runs very well in my linux system, and Chinese can be displayed. The previous ones all reported overflow errors, I don't know why. Finally, thank you very much for your warm help to the novice in these two days. Wish you a happy work!

Code: Pascal  [Select][+][-]
  1. 1.//$>./project1 --part=fang
  2. Output:   jianwt@jianwt-PC:~/Desktop/TestCachewt/TestCache$ ./project1 --part=fang
  3.               Found 245 fonts.
  4.               Name - [FangSong]
  5.               File - [/usr/share/fonts/SIMFANG.TTF]
  6.               Regular
  7.  
  8.               Name - [FangSong]
  9.               File - [/home/jianwt/.fonts/SIMFANG.TTF]
  10.               Regular
  11.  
  12.               Name - [FangSong_GB2312]
  13.               File - [/home/jianwt/.fonts/仿宋_GB2312.ttf]
  14.              Regular
  15.  
  16.            jianwt@jianwt-PC:~/Desktop/TestCachewt/TestCache$
  17.  
  18.  
  19. 2.//$>   cp  ~/.fonts/SIMFANG.TTF  ~/.fonts/simfang.ttf
  20. Output:jianwt@jianwt-PC:~/Desktop/TestCachewt/TestCache$ cp  ~/.fonts/SIMFANG.TTF  ~/.fonts/simfang.ttf
  21.             jianwt@jianwt-PC:~/Desktop/TestCachewt/TestCache$
  22.  
  23. 3.//$> sudo  cp ~/.fonts/simfang.ttf /usr/local/share/fonts/truetype/.
  24. Output: jianwt @ jianwt - PC: ~ / Desktop/TestCachewt TestCache $sudo cp ~ /. / fonts/simfang the vera.ttf/usr/local/share/fonts/truetype /.
  25.             Please enter your password:
  26.             Successful verification
  27.             Cp: unable to create the common file '/ usr/local/share/fonts/truetype /.' : the file or directory
  28.            jianwt@jianwt-PC:~/Desktop/TestCachewt/TestCache$
  29.  
  30.  
  31.  
Title: Re: Chinese Fonts in fpPDF
Post by: paweld on October 03, 2024, 07:05:19 am
Where can I get the file "pdftoppm"? Can you give me a download website?
I use Deian and here it is installed along with the system. You can also download from: https://www.xpdfreader.com/download.html , but here you must already use “pdftopng”, which does not have the “singlefile” parameter
Title: Re: Chinese Fonts in fpPDF
Post by: dbannon on October 03, 2024, 09:13:42 am
Your last TestPDF runs very well in my linux system, and Chinese can be displayed.

Yippy, we've got it at last !

Quote
The previous ones all reported overflow errors,
That was because of the Latin fonts I, incorrectly, assumed were present on all systems.  So, all along you could show chinese fonts, but my code used some English fonts as well, they were the problem. Sigh ...

Thanks for not giving up jianwt.

Davo
Title: Re: Chinese Fonts in fpPDF
Post by: jianwt on October 03, 2024, 10:05:42 am
@paweld

@dbannon

Thanks to the friends who helped me.
Title: Re: Chinese Fonts in fpPDF
Post by: jianwt on October 03, 2024, 11:02:05 am
@paweld
I'm not really good with this. For example, when I use the Xpdf command line tools in windows system, I put "pdftopng.exe" and "project1.exe" in the same directory. Then, I changed the code according to the method you said. Run project1.exe but don't see the converted image. What's wrong? Please help. thank you.
Do you need anything else?
Do you still need to install the corresponding version (GPG signature)? Could you please help me check whether the code I modified is correct?

Code: Pascal  [Select][+][-]
  1.  // cmd := 'pdftoppm';
  2.  // if not RunCommand(cmd, ['-singlefile', filename + '.pdf', filename, '-png'], outstr, [poWaitOnExit], swoHIDE) or not FileExists(filename + '.png') then
  3.   //  exit;
  4.  
  5.   cmd := 'pdftopng';
  6.   if not RunCommand(cmd, [ filename, '-png'], outstr, [poWaitOnExit], swoHIDE) or not FileExists(filename + '.png') then
  7.     exit;        
  8.  
Title: Re: Chinese Fonts in fpPDF
Post by: paweld on October 03, 2024, 04:17:28 pm
if you want to use pdftopng then you need to modify the code a bit, to something like the following:
Code: Pascal  [Select][+][-]
  1.       cmd := {$IFDEF WINDOWS}'pdftopng.exe'{$ELSE}'pdftopng'{$ENDIF};
  2.       //run PDFTOPNG with parameters -f NUMBER_OF_FIRST_PAGE -l NUMBER_OF_LASTPAGE INPUT_PDF_FILENAME OUTPUT_PNG_FILENAME_WITHOUT_EXT
  3.       if not RunCommand(cmd, ['-f', '1', '-l', '1', filename + '.pdf', filename], outstr, [poWaitOnExit], swoHIDE) or not FileExists(filename + '-000001.png') then
  4.         exit;
  5.       Result := TMemoryStream.Create;
  6.       Result.LoadFromFile(filename + '-000001.png');
  7.       DeleteFile(filename + '.pdf');
  8.       DeleteFile(filename + '-000001.png');
It works on Windows and Linux, but under Windows at my place the preview for Chinese characters is not generated correctly.

However, as for RunCommand, it is a function that calls external commands - you need to provide the command (program), a list of parameters as an array, then there is an output string, run options (poWaitOnExit - wait for program execution) and window options (swoHide - hide window)
Title: Re: Chinese Fonts in fpPDF
Post by: jianwt on October 04, 2024, 03:25:25 am
@paweld
That's it. Thank you very much.
Title: Re: Chinese Fonts in fpPDF
Post by: dbannon on October 04, 2024, 05:13:21 am
If you are trying to open that pdf and read it from a FPC/Lazarus program, maybe you would be better using the Operating System's ability to open a given file with the appropriate application.  Assuming you (or your end user) system is configured with an application that can open a PDF, you should use it.

This code, from my app, takes a full path to a file name and tries to open it. It does not need to know what application to use, it just asks the operating system to open it. You would probably be passing only PDF ?

Code: Pascal  [Select][+][-]
  1. uses     FileUtil, LCLIntf;     // probably a few more needed here ...
  2. ....
  3. function OpenFileLink(LinkText : string) : boolean;
  4. var
  5.     i : integer;
  6.     Msg : string;
  7.     {$ifdef WINDOWS}        // in Windows, any file is potentially executable, we'll test by extension
  8.     function WindowsFileIsExecutable() : boolean;                          // True if file has extension mentioned in PATHEXT
  9.     var WinExeExt, Extension : string;
  10.     begin
  11.         WinExeExt := GetEnvironmentVariable('PATHEXT');
  12.         // WinExeExt := '.COM;.EXE;.BAT;.CMD;.VBS;.VBE;.JS;.JSE;.WSF;.WSH';   // just for testing purposes !!!!
  13.         if WinExeExt = '' then
  14.             exit(false);        // var not set, we will not test or warn for executable
  15.         WinExeExt := WinExeExt + ';';
  16.         Extension := ExtractFileExt(LinkText);
  17.         if Extension = '' then
  18.             exit(False);        // Again, not enough info, will not test or warn for executable
  19.         Extension := uppercase(Extension) + ';';
  20.         Result := (pos(Extension, WinExeExt) <> 0);
  21.     end;
  22.     {$endif}
  23.  
  24. begin
  25.     result := True;
  26.     if LinkText.StartsWith(FileLinkToken) then
  27.         LinkText := LinkText.Remove(0, FileLinkTokenLen)
  28.     else exit;                                                // thats an error, should not happen.
  29.     if (LinkText = '') or (LinkText = '""') then begin
  30.         showmessage(' Empty Link ');
  31.         exit;
  32.     end;                                                      // OK, we know its not empty, but does it use " ?
  33.     if (LinkText[1] = '"') then begin                         // Ah, wrapped in ".."
  34.         LinkText := LinkText.Remove(0, 1);                    // Lazarus code will re-wrap the text later in process
  35.         i := LinkText.IndexOf('"', 0);                        // Must have a second "
  36.         if i = -1 then begin
  37.             showmessage('Badly formed link : '#10 + LinkText);
  38.             exit;
  39.         end;
  40.         LinkText := LinkText.Remove(i, 99);                   // remove second " and anything after it too
  41.     end;
  42.     if LinkText = '' then begin                               // Probably redundant .....
  43.         showmessage('Empty Link');
  44.         exit;
  45.     end;
  46.  
  47.     { its not an absolute path (and therefore needs $HOME) if it does not start with a slash or eg c:/
  48.     So, prepend $HOME if it does not start with either a slash or ?:  !     }
  49.     // Is it an absolute path, if not, prepend $HOME
  50.     if not (LinkText[1] in ['\', '/']) then begin         // Relative path if first char after token is not a slash
  51.         {$ifdef WINDOWS}                                  // it might still be an absolute path, starts with eg c:\ ?
  52.         if (length(LinkText) < 4)                         // too short
  53.             or (LinkText[2] <> ':') then                  // not a drive specifier, not much of a test but its windows !
  54.                 LinkText := Sett.HomeDir + LinkText;
  55.         {$else}
  56.         LinkText := Sett.HomeDir + LinkText;
  57.         {$endif}
  58.     end;
  59.     if not (FileExists(LinkText) or DirectoryExists(LinkText)) then begin
  60.         showmessage('File does not exist : '#10 + LinkText);
  61.         exit;
  62.     end;
  63.  
  64.         {$ifdef WINDOWS}
  65.         // 'Executable' on Windows is a dogs breakfast. https://forum.lazarus.freepascal.org/index.php/topic,24615.0.html
  66.         if WindowsFileIsExecutable() then begin
  67.         {$else}
  68.         if FileIsExecutable(LinkText) then begin
  69.         {$endif}
  70.              Msg := 'Link is an executable file.';
  71.              if IDYES <> Application.MessageBox('Open an executable ?'
  72.                      ,pchar(Msg) , MB_ICONQUESTION + MB_YESNO) then
  73.                  exit;
  74.          end;
  75.         if not OpenDocument(LinkText) then
  76.              showmessage('Sorry, cannot open '#10 + LinkText);
  77.  
  78. end;  
                   

edit : guess at extra uses units needed
Title: Re: Chinese Fonts in fpPDF
Post by: paweld on October 04, 2024, 07:48:09 am
@dbannon: in LCLIntf unit there are functions OpenDocument and OpenURL
Title: Re: Chinese Fonts in fpPDF
Post by: dbannon on October 04, 2024, 10:00:53 am
@dbannon: in LCLIntf unit there are functions OpenDocument and OpenURL

Yep, the code about, eventually, calls OpenDocument().  But does need to do some checks first. This particularly so with Windows that does not have a clear idea of 'executable'.

Davo
Title: Re: Chinese Fonts in fpPDF
Post by: jianwt on October 05, 2024, 01:53:14 pm
First of all, thanks to 'paweld' for writing this demo, I wanted to write a simple complete routine for fpPDF.

Now there is a problem: how to accurately base (TPDFpage.paper.w - left margin - right margin), and then write the text line into the TPDFPage work area. At present, there are problems in the function WrapLines(), one is that the overall content of SynEdit1.Text should be pressed to enter the paragraph branch, rather than the content branch of SynEdit1.Lines[J], and the second is that the branch calculation is inaccurate when encountering Chinese branches.

I am a novice, please help to complete this demo. Thank you all in advance.
Title: Re: Chinese Fonts in fpPDF
Post by: paweld on October 05, 2024, 03:19:01 pm
This error I corrected, but the issue of too narrow text remains
Code: Pascal  [Select][+][-]
  1. procedure TForm1.WrapLines(aStr: String; aPageWidth, aFontSize: Single; var aLines: TStringList);
  2. var
  3.   textwidth: Single;
  4.   stmp: String;
  5.  
  6.   function Inseparable(aCheckStr: String): Boolean;
  7.   const
  8.     icarr: array [0..10] of char = ' ,.;/\+-|?!';
  9.   var
  10.     ii, ip, il: Integer;
  11.   begin
  12.     Result := True;
  13.     il := UTF8Length(aCheckStr);
  14.     for ii := 0 to High(icarr) do
  15.     begin
  16.       ip := UTF8Pos(icarr[ii], aCheckStr);
  17.       if (ip > 0) and (ip < il) then
  18.       begin
  19.         Result := False;
  20.         break;
  21.       end;
  22.     end;
  23.   end;
  24.  
  25.   function SeparatorPos(aCheckStr: String; startfrom: Integer; First: Boolean): Integer;   //first or last separator in string
  26.   const
  27.     icarr: array [0..10] of char = ' ,.;/\+-|?!';
  28.   var
  29.     ii, ip, il: Integer;
  30.     b: Boolean;
  31.   begin
  32.     if startfrom < 1 then
  33.       startfrom := 1;
  34.     il := UTF8Length(aCheckStr);
  35.     Result := UTF8Length(aCheckStr);
  36.     b := False;
  37.     if not First then
  38.     begin
  39.       aCheckStr := UTF8ReverseString(aCheckStr);
  40.       ii := 1;
  41.       while (Length(aCheckStr) > ii) and (aCheckStr[ii] in [' ', ',', '.', ';', '/', '\', '+', '-', '|', '?', '!']) do
  42.         Inc(ii);
  43.       if startfrom < ii then
  44.         startfrom := ii;
  45.     end;
  46.     for ii := 0 to High(icarr) do
  47.     begin
  48.       ip := UTF8Pos(icarr[ii], aCheckStr, startfrom);
  49.       if (ip > 0) and (ip < Result) then
  50.       begin
  51.         Result := ip;
  52.         b := True;
  53.       end;
  54.     end;
  55.     if not First and b then
  56.       Result := il + 1 - Result;
  57.   end;
  58.  
  59. begin
  60.   aLines.Clear;
  61.   textwidth := PdfToMm(fci.TextWidth(aStr, aFontSize));
  62.   //if inseparable text
  63.   if (aStr = '') or (textwidth <= aPageWidth) or Inseparable(aStr) then
  64.   begin
  65.     textwidth := 0;
  66.     if aStr = '' then
  67.       aStr := ' ';
  68.     aLines.Add(aStr);
  69.     aStr := '';
  70.   end;
  71.   while textwidth > aPageWidth do
  72.   begin
  73.     stmp := TrimRight(UTF8Copy(aStr, 1, SeparatorPos(aStr, trunc(RoundTo(UTF8Length(aStr) * aPageWidth / textwidth, 0)), True)));
  74.     if Inseparable(stmp) then
  75.       stmp := UTF8Copy(aStr, 1, SeparatorPos(aStr, 1, True))
  76.     else
  77.     begin
  78.       stmp := TrimRight(UTF8Copy(stmp, 1, SeparatorPos(stmp, 1, False)));
  79.       while (PdfToMm(fci.TextWidth(stmp, aFontSize)) > aPageWidth) and not Inseparable(stmp) do
  80.         stmp := TrimRight(UTF8Copy(stmp, 1, SeparatorPos(stmp, 1, False)));
  81.     end;
  82.     aStr := UTF8Copy(aStr, UTF8Length(stmp) + 1, UTF8Length(aStr) - UTF8Length(stmp));
  83.     if (aStr <> '') and (aStr[1] = ' ') then
  84.       Delete(aStr, 1, 1);
  85.     if stmp <> '' then
  86.       aLines.Add(stmp);
  87.     textwidth := PdfToMm(fci.TextWidth(aStr, aFontSize));
  88.   end;
  89.   if aStr <> '' then
  90.     aLines.Add(aStr);
  91. end;    
  92.  
Title: Re: Chinese Fonts in fpPDF
Post by: jianwt on October 06, 2024, 07:34:56 am
@paweld
There are still some issues to be addressed:
1.   SynEdit1.  Text needs to be split into lines based on the entire content, with specific requirements.   First, the number of paragraphs in SynEdit1.  Text needs to be calculated, and then each paragraph needs to be split into lines.   It is not possible to split lines using SynEdit1.Lines[J].
2.   The inaccurate calculation results are due to the fact that the parameter aPageWidth in the WrapLines() function is in millimeters, while textwidth := PdfToMm(fci.  TextWidth(aStr, aFontSize)) returns a numerical value in pixels.   They must be consistent in their unit of measurement.
3.   The function WrapLines() splits the text into paragraphs when it encounters a space in SynEdit1.  Text, which is incorrect.
4.   In Chinese writing tradition, the first character of each paragraph should be indented by 2 Chinese characters.
(The Internet translation may be mistranslated.)
See the diagram below for details:
Title: Re: Chinese Fonts in fpPDF
Post by: jianwt on October 07, 2024, 10:07:44 am
@dbannon
Actually, I originally wanted to upload a part of the files separated from your project, but since I didn't get your consent, I didn't upload those program files here.
It seems that my version of KMEMOA is not quite the same as yours, some constants or variables I can't run, very strange oh?

My idea is to generate a PDF document that matches the display effect of the "Chinese_editing_format_in_WORD.doc" document in the project using a method.  The text content for generating the document comes from KMEMO, and the illustrations are from the image "Inserted_picture.jpg" in the project folder.  Additionally, page numbers should be clearly marked at the bottom of each page (Page 1...  Page N) to make it perfect.

Due to the forum restriction that the attachment cannot be larger than 500KB, I have compressed the document in separate volumes. Please download the two compressed reports and then decompress them.

zip Package 1:
Title: Re: Chinese Fonts in fpPDF
Post by: jianwt on October 07, 2024, 10:16:43 am
zip Package 2:
Please download the "zip Package 2:" attachment and rename it from "Chinese_editing_format_in_WORD2.zip" to "Chinese_editing_format_in_WORD.z01". Then, extract it along with the "zip Package 1:" file to obtain the Word document.
Title: Re: Chinese Fonts in fpPDF
Post by: dbannon on October 12, 2024, 11:06:08 am
Jianwt, sorry for the delay, attached is the zip with another project, again based on the original example. As we discussed in the PM.

It contains a text file that, I think, is what you sent me as Chinese Characters. Its called c.txt and might contain a utf8 error so, probably worth looking at it and see if it all makes sense.  Sadly, I cannot read a single Chinese character so I really don't know.

I have chose some silly font setting and layout numbers so that it spreads over three pages because I understand you want to know how to put page numbers in. Easy for you to change those, I am sure you will want to.

Now, layout, I do not know the correct way to layout Chinese text, for that I apologize but it is what it is.  I have assumed that you just put the text into a block, separated by new-lines. Please run it and see if this is what you expect. If not, it should be easy for you to see what I have got wrong !

All the best,

Davo
TinyPortal © 2005-2018