Lazarus

Free Pascal => Beginners => Topic started by: Nick on November 21, 2015, 11:16:39 am

Title: 3.0 RC2, Windows: non-unicode API by default?
Post by: Nick on November 21, 2015, 11:16:39 am
I am just beginner in FPC (but some advanced in developing), so... :)
NTFS file-system is unicode, so there might be files with unicode characters, and are there, on people's disks. But we can't use them with default FPC settings.
If program uses TStringList.LoadFromFile then user get error on open file. Example: windows with US locale, and file name like "Имя в Юникоде.txt".
As a workaround we can rebuild fpc-libraries with FPC_OS_UNICODE, but this is workaround.
I think it will be useful to have unicode libraries too, in distro.
Title: Re: 3.0 RC2, Windows: non-unicode API by default?
Post by: Thaddy on November 21, 2015, 04:22:43 pm
I don't understand the problem:
FPC has about the richest set of string types available and to be more specific NTFS has never been any problem under Windows. There really is no need to recompile the whole with FPC_OS_UNICODE and it is not a workaround in that sense. The Unicode API's are fully accessible by means of the UnicodeString type, which is a Windows compatible UTF16 reference counted string type compatible with Delphi or C#.

Can you give us a small example where you ran into trouble?

I guess you are simply looking for a modeswitch ;) and indeed that is there:
Use {$modeswitch unicodestrings} or even better (since you are developing for Windows) use the {$mode delphiunicode} which switches the default string to a Delphi 2009+ compatible Unicode16 string.
"string" becomes an alias for "unicodestring"in these cases and the "XxxxW" windows Api's are called.

Also, Lazarus has an intermediate solution based on UTF8 by default.

If you ran into trouble, the most likely that happened is insufficient reading of the very good documentation.

There ARE good reasons to recompile with FPC_OS_UNICODE but these are not specifically related to Windows API's or NTFS (which is a special case of the former).
Title: Re: 3.0 RC2, Windows: non-unicode API by default?
Post by: Nick on November 21, 2015, 06:28:00 pm
I checked it out.
Code: Pascal  [Select][+][-]
  1. program UnicodeTest1;
  2.  
  3. {$H+}
  4. {$mode delphiunicode}
  5. {$modeswitch unicodestrings}
  6. {$define Unicode}
  7. {$define FPC_OS_UNICODE}
  8.  
  9. uses
  10.   Classes, SysUtils, CustApp,
  11.   Windows;
  12.  
  13. type
  14.   TMyApplication = class(TCustomApplication)
  15.   protected
  16.     procedure DoRun; override;
  17.   end;
  18.  
  19. procedure CreateFileA; external 'kernel32' name 'CreateFileA';
  20. //procedure CreateFileW; external 'kernel32' name 'CreateFileW';
  21.  
  22. { TMyApplication }
  23.  
  24. procedure TMyApplication.DoRun;
  25. begin
  26.   if @CreateFile = @CreateFileA then
  27.     WriteLn('Ansi API')
  28.   else
  29.     WriteLn('Unicode API');
  30.  
  31. //  WriteLn('SizeOf(Char) = ' + IntToStr(SizeOf(Char)));
  32.  
  33.   WriteLn('Press enter...');
  34.   Readln;
  35.   // stop program loop
  36.   Terminate;
  37. end;
  38.  
  39. var
  40.   Application: TMyApplication;
  41. begin
  42.   Application:=TMyApplication.Create(nil);
  43.   Application.Title:='My Application';
  44.   Application.Run;
  45.   Application.Free;
  46. end.
  47.  
Result: Ansi API
Title: Re: 3.0 RC2, Windows: non-unicode API by default?
Post by: marcov on November 21, 2015, 07:41:48 pm
Quote
  if @CreateFile = @CreateFileA then
    WriteLn('Ansi API')
  else
    WriteLn('Unicode API');

As the person that created FPC_OS_UNICODE, I still find it hard to understand the actual problem.

Whatever "createfile" is aliased too doesn't really matter much. Changing it to _W won't make everything automatically wide tolerant.3

So maybe take a step back and describe what is actually the problem.
Title: Re: 3.0 RC2, Windows: non-unicode API by default?
Post by: Arvur on October 05, 2017, 09:50:08 am
This seems to be my case. First - the problem. I need compatibility with Delphi code (XE6) without massive modifications.

Documentation states it is real. So I set {$mode delphiunicode} (in project options and in the unit) and ... get lots of errors. They are of two types generally:

1. Error: Incompatible type for arg no. 6: Got "PWideChar", expected "PChar"
Code: Pascal  [Select][+][-]
  1. function SearchFilePath(const FName: string) : string;
  2. var
  3.   i, Len: Integer;
  4.   P: PChar;
  5. begin
  6.   i := SearchPath(nil, PChar(FName), nil, Len, PChar(Result), P);
  7.  
I dig into SearchPath and see external 'kernel32' name 'SearchPathA' in ascdef.inc

2. Warning: Implicit string type conversion with potential data loss from "UnicodeString" to "RawByteString"
Code: Pascal  [Select][+][-]
  1. var
  2.   Dir: string;
  3.   SearchRec: TSearchRec;
  4. begin
  5.   if FindFirst(IncludeTrailingPathDelimiter(Dir) + '*.*', faAnyFile, SearchRec) = 0 then
  6.  
FindFirst falls back to its RawByteString version as SearchRec is TRawByteSearchRec (fileutilh.inc)

This way {$mode delphiunicode} is completelly useless. I've tried to add FPC_UNICODE_RTL and FPC_OS_UNICODE to project defines. That does not help.

What is the correct way to use {$mode delphiunicode} to be compatible with Delphi source?

P.S. Lazarus 1.6.4 with FPC 3.0.2
Title: Re: 3.0 RC2, Windows: non-unicode API by default?
Post by: JuhaManninen on October 05, 2017, 12:08:22 pm
What is the correct way to use {$mode delphiunicode} to be compatible with Delphi source?
It is not possible yet. The most Delphi compatible solution now is the UTF-8 solution provided by Lazarus:
 http://wiki.freepascal.org/Unicode_Support_in_Lazarus
Yes, it is very compatible at source level despite the different encodings. Check it out.
Title: Re: 3.0 RC2, Windows: non-unicode API by default?
Post by: Arvur on October 05, 2017, 01:29:09 pm
Yes, it is very compatible at source level despite the different encodings. Check it out.
And the encodings are really painful to this project :D So I'm looking for other ways.

Is rebuilding fpc (https://forum.lazarus.freepascal.org/index.php?topic=31065.0) with unicode defines still not worth trying?
Title: Re: 3.0 RC2, Windows: non-unicode API by default?
Post by: JuhaManninen on October 05, 2017, 01:40:21 pm
And the encodings are really painful to this project :D So I'm looking for other ways.
Give me an example please. I believe you have some kind of prejudice now.
The worst case scenario is that you want to treat UTF-16 as UCS-2, thus creating buggy code.
Did you look at the wiki page? It tells you how to make encoding agnostic code that works in both Lazarus and in Delphi, even if you must iterate codepoints or Unicode "characters".

Quote
Is rebuilding fpc (https://forum.lazarus.freepascal.org/index.php?topic=31065.0) with unicode defines still not worth trying?
No.
Title: Re: 3.0 RC2, Windows: non-unicode API by default?
Post by: Arvur on October 05, 2017, 03:12:53 pm
I believe you have some kind of prejudice now.
Perhapse I was doing something wrong last time. Should give it a try once more.
I'm going to:
- convert all units to UTF8
- use {$mode delphi}
- remove {$H+}
- add -FcUTF8
Is this scenario Ok?
Title: Re: 3.0 RC2, Windows: non-unicode API by default?
Post by: JuhaManninen on October 05, 2017, 03:46:33 pm
- convert all units to UTF8
- use {$mode delphi}
- remove {$H+}
- add -FcUTF8
Is this scenario Ok?
Source files must be UTF-8, yes.
{$H+} is included in {$mode delphi}. Can be removed yes.
-FcUTF8 is usually not needed. Things work better without it. Please read explanation from the wiki page.
Title: Re: 3.0 RC2, Windows: non-unicode API by default?
Post by: Arvur on October 10, 2017, 04:08:23 pm
JuhaManninen, I still have Ansi-version of Windows API (STARTUPINFO is STARTUPINFOA, CreateProcess is CreateProcessA, etc).
Is it supposed that I have to rewrite all API-calls?
Title: Re: 3.0 RC2, Windows: non-unicode API by default?
Post by: Bart on October 10, 2017, 04:12:36 pm
If you call these API's directly by yourself, then yes, rewrite them to use the W-API's.

Bart
Title: Re: 3.0 RC2, Windows: non-unicode API by default?
Post by: Arvur on October 10, 2017, 04:24:00 pm
Then this "compatibility" makes very little sense  :(
Title: Re: 3.0 RC2, Windows: non-unicode API by default?
Post by: Arvur on October 10, 2017, 05:36:05 pm
Nice thing! In Delphi XE6 unit without BOM. String constant in code gives somethings like this
Quote
Не указано имя для создания пары ключей
Unit in UTF8BOM works fine.

This means I'll always have -FcUTF8 if I want my code to be compiled both in Delphi and Lazarus without additional changes.
Title: Re: 3.0 RC2, Windows: non-unicode API by default?
Post by: JuhaManninen on October 10, 2017, 05:39:06 pm
Then this "compatibility" makes very little sense  :(
Could you tell please why you want to use the historical ANSI system codepages instead of Unicode?
Unicode has existed for decades and solves all problems inherent to the codepages.
Title: Re: 3.0 RC2, Windows: non-unicode API by default?
Post by: balazsszekely on October 10, 2017, 05:48:32 pm
Could you tell please why you want to use the historical ANSI system codepages instead of Unicode?
Unicode has existed for decades and solves all problems inherent to the codepages.
If I understood him correctly the API calls are not linked directly to their _W equivalent. When you write CreateProcess is "mapped" to CreateProcessA not CreateProcessW.
Title: Re: 3.0 RC2, Windows: non-unicode API by default?
Post by: JuhaManninen on October 10, 2017, 07:38:42 pm
If I understood him correctly the API calls are not linked directly to their _W equivalent. When you write CreateProcess is "mapped" to CreateProcessA not CreateProcessW.
True, in FPC 3.0.2 sources CreateProcess maps to CreateProcessA. In future the defaults must be changed to use the _W API versions. It is important for Unicode support, regardless of encoding.
TinyPortal © 2005-2018