cross platform text with European accents .

[lazer]

Hi,

I have a program displaying the content of some text files with accented characters. I'm having trouble getting it to display consistently between Windows and Linux.

I seem to need utf8 on Linux and ISO-8859-15 on Windows.

I don't want to have to have two versions of the text files. Is there common encoding  I can use of both platforms?

TIA.

[Lulu]

If I am not mistaken, UTF8 is the 'standard' in Lazarus LCL for all platforms. Its curious that you have to encode your files with ISO-xxxx on Windows. Can you provide some code to show the way you read this file and displaying it ?

[lazer]

Yes, you are correct. utf8 seems to conserve accents.

However, if I open the file with Notepad in Win32 and save it, it is growing by 3 bytes. Since I have a checksum on the file this is causing my program to flag a corrupt file.

I created the text files on linux with featherpad, used unix2dos to add the CRLF and then did my checksum.

I need to find a stable format.

[marcov]

Probably notepad adds a BOM. You can avoid the differences by adding a BOM in the original file (and hope neither featherpad or unix2dos remove it)

[domasz]

Yes, you are correct. utf8 seems to conserve accents.

Install a hex editor. HxD is free and pretty nice. Then you will be able to see differences. In this case those few bytes are UTF-8 BOM.

[lazer]

Code: Pascal  [Select][+][-]

1. echo -en '\xEF\xBB\xBF' > BOM.dat
2.  
3. cat  BOM.dat test.utf8 > test.txt
4.  

Seems to work . Thanks for pointing out that win32 puts a byte order marker in file with single byte data    I would not have guessed.

[dseligo]

I need to find a stable format.

Use XML then and don't do checksum on whole file but only on data.

[dseligo]

win32 puts a byte order marker in file with single byte data

Not win32, but Notepad. There are editors that don't put BOM (automatically) in file.

[lazer]

Notepad is part of the default windows installation since Win3.1  IIRC.  It's Windows.