Lazarus

Using the Lazarus IDE => General => Topic started by: crazydog on August 05, 2017, 04:47:19 am

Title: Writing Chinese Characters on text
Post by: crazydog on August 05, 2017, 04:47:19 am
How can I write Chinese Characters on a .txt file?
when I use the rewrite function,
the Chinese Characters become gibberish on the .txt

eg
中學部(in Program) >>>>> 銝剖飛??(In Text)
Title: Re: Writing Chinese Characters on text
Post by: engkin on August 05, 2017, 06:02:42 am
Show the code you used.

What did you use to see or open the txt file, Notepad?

What OS/Lazarus/FPC version?

Meanwhile the following code gives the attached text file:
Code: Pascal  [Select][+][-]
  1. program chtest;
  2. {$mode objfpc}{$H+}
  3.  
  4. uses
  5.   Classes;
  6.  
  7. var
  8.   sl: TStringList;
  9.  
  10. begin
  11.   sl := TStringList.Create;
  12.   try
  13.     sl.Add('中學部');
  14.     sl.SaveToFile('ch-test.txt');
  15.   finally
  16.     sl.Free;
  17.   end;
  18. end.
Title: Re: Writing Chinese Characters on text
Post by: crazydog on August 06, 2017, 08:24:06 am
Show the code you used.

What did you use to see or open the txt file, Notepad?

What OS/Lazarus/FPC version?

Meanwhile the following code gives the attached text file:
Code: Pascal  [Select][+][-]
  1. program chtest;
  2. {$mode objfpc}{$H+}
  3.  
  4. uses
  5.   Classes;
  6.  
  7. var
  8.   sl: TStringList;
  9.  
  10. begin
  11.   sl := TStringList.Create;
  12.   try
  13.     sl.Add('中學部');
  14.     sl.SaveToFile('ch-test.txt');
  15.   finally
  16.     sl.Free;
  17.   end;
  18. end.
I'm using Lazarus IDE v1.4.4
Here's my Code,
It seems that it worked for txt, (didnt realize it b4)
but when it is .csv and opened by Microsoft Office Excel,
'中學部' turns into '銝剖飛??'
Code: Pascal  [Select][+][-]
  1. procedure AddData;
  2. var Data:TextFile;
  3. begin
  4.   assign(Data,'Data.csv');
  5.   rewrite(Data);
  6.   write(Data,'中學部');
  7.   Closefile(Data)
  8. end;          
Title: Re: Writing Chinese Characters on text
Post by: Zath on August 06, 2017, 10:13:54 am
Is it changing it to a different version of Chinese ?
Simplified and traditional are often available but maybe not to all applications on your system.
Title: Re: Writing Chinese Characters on text
Post by: engkin on August 06, 2017, 05:47:03 pm
I'm using Lazarus IDE v1.4.4
Here's my Code,
It seems that it worked for txt, (didnt realize it b4)
but when it is .csv and opened by Microsoft Office Excel,
'中學部' turns into '銝剖飛??'
Code: Pascal  [Select][+][-]
  1. procedure AddData;
  2. var Data:TextFile;
  3. begin
  4.   assign(Data,'Data.csv');
  5.   rewrite(Data);
  6.   write(Data,'中學部');
  7.   Closefile(Data)
  8. end;          

That's an Excel bug (or feature). It considers the csv file encoding to have the same encoding as your system (which is not true, your system has an ansi encoding, while the file is UTF8).

A workaround is to use UTF16LE encoding with BOM marker.

Edit:
Simple test:
Code: Pascal  [Select][+][-]
  1. uses
  2.   LazUTF8,...;
  3. ...
  4. procedure AddData;
  5. var Data:File;
  6.   s: String;
  7.   ss: WideString;  { or UnicodeString }
  8. begin
  9.   assign(Data,'Data.csv');
  10.   rewrite(Data);
  11.   Reset(Data, 1);
  12.   BlockWrite(Data, #$FF#$FE, 2);  { Write BOM marker }
  13.   s := '中學部';  { UTF8 }
  14.   ss := s; { or ss := UTF8ToUTF16(s); } { UTF16 }
  15.   BlockWrite(Data, ss[1], Length(RawByteString(ss)));
  16.   Closefile(Data)
  17. end;
Title: Re: Writing Chinese Characters on text
Post by: crazydog on August 09, 2017, 02:25:31 pm
I'm using Lazarus IDE v1.4.4
Here's my Code,
It seems that it worked for txt, (didnt realize it b4)
but when it is .csv and opened by Microsoft Office Excel,
'中學部' turns into '銝剖飛??'
Code: Pascal  [Select][+][-]
  1. procedure AddData;
  2. var Data:TextFile;
  3. begin
  4.   assign(Data,'Data.csv');
  5.   rewrite(Data);
  6.   write(Data,'中學部');
  7.   Closefile(Data)
  8. end;          

That's an Excel bug (or feature). It considers the csv file encoding to have the same encoding as your system (which is not true, your system has an ansi encoding, while the file is UTF8).

A workaround is to use UTF16LE encoding with BOM marker.

Edit:
Simple test:
Code: Pascal  [Select][+][-]
  1. uses
  2.   LazUTF8,...;
  3. ...
  4. procedure AddData;
  5. var Data:File;
  6.   s: String;
  7.   ss: WideString;  { or UnicodeString }
  8. begin
  9.   assign(Data,'Data.csv');
  10.   rewrite(Data);
  11.   Reset(Data, 1);
  12.   BlockWrite(Data, #$FF#$FE, 2);  { Write BOM marker }
  13.   s := '中學部';  { UTF8 }
  14.   ss := s; { or ss := UTF8ToUTF16(s); } { UTF16 }
  15.   BlockWrite(Data, ss[1], Length(RawByteString(ss)));
  16.   Closefile(Data)
  17. end;
Thx for advice :D
it seems that it is the default encoding problem
so I changed the default encoding of my system to UTF8
and it worked fine now
btw when I use ur program,
It shows Identifier not found "RawByteString",
is anything missing?
Title: Re: Writing Chinese Characters on text
Post by: engkin on August 09, 2017, 04:01:03 pm
Thx for advice :D
it seems that it is the default encoding problem
so I changed the default encoding of my system to UTF8
and it worked fine now

Glad it worked for you. The other day I was reading that Excel on Mac does not seem to correctly handle UTF8-encoded CSV files. That's why I mentioned the other encoding. If it is for personal use then you should be ok.

btw when I use ur program,
It shows Identifier not found "RawByteString",
is anything missing?

RawByteString (https://www.freepascal.org/docs-html/rtl/system/rawbytestring.html) is there for a while now. You need a recent Lazarus/FPC to use it.
TinyPortal © 2005-2018