Strange implicit string conversion

[Sergey Tkachenko]

Let we have two simple functions (Lazarus 3.8 for Windows):

Code: Pascal  [Select][+][-]

1. function DoubleS(const s: String): String;
2. begin
3.   Result := S+S;
4. end;
5.  
6. function SingleS(const s: String): String;
7. begin
8.   Result := S;
9. end;

Let's test on the string

Code: Pascal  [Select][+][-]

1. var
2.   S: String;
3.  
4.   S := 'αβγ';

1)

Code: Pascal  [Select][+][-]

1.   Caption := SingleS(S);

Result 'αβγ'. OK.

2)

Code: Pascal  [Select][+][-]

1.   Caption := DoubleS(S);

Result 'αβγαβγ'. OK.

3)

Code: Pascal  [Select][+][-]

1.   S := Utf8Encode(UTF8Decode(S));
2.   Caption := SingleS(S);

Result 'αβγ'. OK.

4)

Code: Pascal  [Select][+][-]

1.   S := Utf8Encode(UTF8Decode(S));
2.   Caption := DoubleS(S);

Result '??????'.
Is there are workaround to fix it?

[Sergey Tkachenko]

Ok, I found a workaround,
S := PChar(Utf8Encode(UTF8Decode(S)));
instead of
S := Utf8Encode(UTF8Decode(S));

But is there more elegant solution?

[Thaddy]

This is not a complete program, we do not see what mode you have specified and hence we do not know the string type in the first place. You also do not mention the compiler version, just a Lazarus version and that is screen drawing and not programming.

[alpine]

@Sergey
Utf8Encode returns RawByteString then DoubleS does concatenation.
Here is the explanation: https://forum.lazarus.freepascal.org/index.php/topic,59958.msg447745.html#msg447745

[Thaddy]

leaving out the pchar cast would help looking more elegant.

[Sergey Tkachenko]

It seems that I need to replace all Utf8Encode functions in my project to

Code: Pascal  [Select][+][-]

1. function MyUtf8Encode(const S: UnicodeString): String;
2. var
3.   SR: RawByteString;
4. begin
5.   SR := UTF8Encode(S);
6.   SetCodePage(SR, 0, False);
7.   Result := SR;
8. end;

[ASerge]

Result '??????'.
Is there are workaround to fix it?

Not reproduced.

Code: Pascal  [Select][+][-]

1. unit Unit1;
2.  
3. {$mode objfpc}{$H+}
4.  
5. interface
6.  
7. uses
8.   Classes, SysUtils, Forms, Controls, Graphics, Dialogs, StdCtrls;
9.  
10. type
11.   TForm1 = class(TForm)
12.     Button1: TButton;
13.     Memo1: TMemo;
14.     procedure Button1Click(Sender: TObject);
15.   private
16.  
17.   public
18.  
19.   end;
20.  
21. var
22.   Form1: TForm1;
23.  
24. implementation
25.  
26. {$R *.lfm}
27.  
28. function DoubleS(const s: string): string;
29. begin
30.   Result := S+S;
31. end;
32.  
33. function SingleS(const s: string): string;
34. begin
35.   Result := S;
36. end;
37.  
38. { TForm1 }
39.  
40. procedure TForm1.Button1Click(Sender: TObject);
41. var
42.   S: String;
43. begin
44.   S := 'αβγ';
45.   Memo1.Append(SingleS(S));
46.   Memo1.Append(DoubleS(S));
47.   S := Utf8Encode(UTF8Decode(S));
48.   Memo1.Append(SingleS(S));
49.   S := Utf8Encode(UTF8Decode(S));
50.   Memo1.Append(SingleS(S));
51. end;
52.  
53. end.

[Sergey Tkachenko]

I found the culprit.
It happens if the project uses fpwidestring unit.

I do not know why I included this unit, so I'll remove it.

[nanobit]

S := Utf8Encode(UTF8Decode(S));

utf8Decode() has a (justifiable) limitation:
it presumes utf8, regardless of actual given codepage.