* * *

Author Topic: Writing Chinese Characters on text  (Read 1127 times)

crazydog

  • New member
  • *
  • Posts: 6
Writing Chinese Characters on text
« on: August 05, 2017, 04:47:19 am »
How can I write Chinese Characters on a .txt file?
when I use the rewrite function,
the Chinese Characters become gibberish on the .txt

eg
中學部(in Program) >>>>> 銝剖飛??(In Text)

engkin

  • Hero Member
  • *****
  • Posts: 1759
Re: Writing Chinese Characters on text
« Reply #1 on: August 05, 2017, 06:02:42 am »
Show the code you used.

What did you use to see or open the txt file, Notepad?

What OS/Lazarus/FPC version?

Meanwhile the following code gives the attached text file:
Code: Pascal  [Select]
  1. program chtest;
  2. {$mode objfpc}{$H+}
  3.  
  4. uses
  5.   Classes;
  6.  
  7. var
  8.   sl: TStringList;
  9.  
  10. begin
  11.   sl := TStringList.Create;
  12.   try
  13.     sl.Add('中學部');
  14.     sl.SaveToFile('ch-test.txt');
  15.   finally
  16.     sl.Free;
  17.   end;
  18. end.
« Last Edit: August 05, 2017, 06:10:44 am by engkin »

crazydog

  • New member
  • *
  • Posts: 6
Re: Writing Chinese Characters on text
« Reply #2 on: August 06, 2017, 08:24:06 am »
Show the code you used.

What did you use to see or open the txt file, Notepad?

What OS/Lazarus/FPC version?

Meanwhile the following code gives the attached text file:
Code: Pascal  [Select]
  1. program chtest;
  2. {$mode objfpc}{$H+}
  3.  
  4. uses
  5.   Classes;
  6.  
  7. var
  8.   sl: TStringList;
  9.  
  10. begin
  11.   sl := TStringList.Create;
  12.   try
  13.     sl.Add('中學部');
  14.     sl.SaveToFile('ch-test.txt');
  15.   finally
  16.     sl.Free;
  17.   end;
  18. end.
I'm using Lazarus IDE v1.4.4
Here's my Code,
It seems that it worked for txt, (didnt realize it b4)
but when it is .csv and opened by Microsoft Office Excel,
'中學部' turns into '銝剖飛??'
Code: Pascal  [Select]
  1. procedure AddData;
  2. var Data:TextFile;
  3. begin
  4.   assign(Data,'Data.csv');
  5.   rewrite(Data);
  6.   write(Data,'中學部');
  7.   Closefile(Data)
  8. end;          

Zath

  • Sr. Member
  • ****
  • Posts: 262
Re: Writing Chinese Characters on text
« Reply #3 on: August 06, 2017, 10:13:54 am »
Is it changing it to a different version of Chinese ?
Simplified and traditional are often available but maybe not to all applications on your system.

engkin

  • Hero Member
  • *****
  • Posts: 1759
Re: Writing Chinese Characters on text
« Reply #4 on: August 06, 2017, 05:47:03 pm »
I'm using Lazarus IDE v1.4.4
Here's my Code,
It seems that it worked for txt, (didnt realize it b4)
but when it is .csv and opened by Microsoft Office Excel,
'中學部' turns into '銝剖飛??'
Code: Pascal  [Select]
  1. procedure AddData;
  2. var Data:TextFile;
  3. begin
  4.   assign(Data,'Data.csv');
  5.   rewrite(Data);
  6.   write(Data,'中學部');
  7.   Closefile(Data)
  8. end;          

That's an Excel bug (or feature). It considers the csv file encoding to have the same encoding as your system (which is not true, your system has an ansi encoding, while the file is UTF8).

A workaround is to use UTF16LE encoding with BOM marker.

Edit:
Simple test:
Code: Pascal  [Select]
  1. uses
  2.   LazUTF8,...;
  3. ...
  4. procedure AddData;
  5. var Data:File;
  6.   s: String;
  7.   ss: WideString;  { or UnicodeString }
  8. begin
  9.   assign(Data,'Data.csv');
  10.   rewrite(Data);
  11.   Reset(Data, 1);
  12.   BlockWrite(Data, #$FF#$FE, 2);  { Write BOM marker }
  13.   s := '中學部';  { UTF8 }
  14.   ss := s; { or ss := UTF8ToUTF16(s); } { UTF16 }
  15.   BlockWrite(Data, ss[1], Length(RawByteString(ss)));
  16.   Closefile(Data)
  17. end;
« Last Edit: August 06, 2017, 06:17:37 pm by engkin »

crazydog

  • New member
  • *
  • Posts: 6
Re: Writing Chinese Characters on text
« Reply #5 on: August 09, 2017, 02:25:31 pm »
I'm using Lazarus IDE v1.4.4
Here's my Code,
It seems that it worked for txt, (didnt realize it b4)
but when it is .csv and opened by Microsoft Office Excel,
'中學部' turns into '銝剖飛??'
Code: Pascal  [Select]
  1. procedure AddData;
  2. var Data:TextFile;
  3. begin
  4.   assign(Data,'Data.csv');
  5.   rewrite(Data);
  6.   write(Data,'中學部');
  7.   Closefile(Data)
  8. end;          

That's an Excel bug (or feature). It considers the csv file encoding to have the same encoding as your system (which is not true, your system has an ansi encoding, while the file is UTF8).

A workaround is to use UTF16LE encoding with BOM marker.

Edit:
Simple test:
Code: Pascal  [Select]
  1. uses
  2.   LazUTF8,...;
  3. ...
  4. procedure AddData;
  5. var Data:File;
  6.   s: String;
  7.   ss: WideString;  { or UnicodeString }
  8. begin
  9.   assign(Data,'Data.csv');
  10.   rewrite(Data);
  11.   Reset(Data, 1);
  12.   BlockWrite(Data, #$FF#$FE, 2);  { Write BOM marker }
  13.   s := '中學部';  { UTF8 }
  14.   ss := s; { or ss := UTF8ToUTF16(s); } { UTF16 }
  15.   BlockWrite(Data, ss[1], Length(RawByteString(ss)));
  16.   Closefile(Data)
  17. end;
Thx for advice :D
it seems that it is the default encoding problem
so I changed the default encoding of my system to UTF8
and it worked fine now
btw when I use ur program,
It shows Identifier not found "RawByteString",
is anything missing?

engkin

  • Hero Member
  • *****
  • Posts: 1759
Re: Writing Chinese Characters on text
« Reply #6 on: August 09, 2017, 04:01:03 pm »
Thx for advice :D
it seems that it is the default encoding problem
so I changed the default encoding of my system to UTF8
and it worked fine now

Glad it worked for you. The other day I was reading that Excel on Mac does not seem to correctly handle UTF8-encoded CSV files. That's why I mentioned the other encoding. If it is for personal use then you should be ok.

btw when I use ur program,
It shows Identifier not found "RawByteString",
is anything missing?

RawByteString is there for a while now. You need a recent Lazarus/FPC to use it.

 

Recent

Get Lazarus at SourceForge.net. Fast, secure and Free Open Source software downloads Open Hub project report for Lazarus