String error

[Jo]

Using svn 14579 0.9.25 (2008-03-19) i386 carbon with Leopard 10.5.2.

I transferred few files from Delphi/windows. I have the following record:

{$PACKRECORDS 1}
 TSyndRec=record
   ID:integer;
   Name:string[150];
   Tipul:array[0..1600] of char;
   Etiology:array[0..2450] of char;
   Tzuna:array[0..1200] of char;
   Shita:array[0..1450] of char;
   Pathology:array[0..3000] of char;
   kat:byte;
   rem:string[10];
   Admin,IsFromUser:boolean;
 end;        

One function I use to read the file is:

function GetPathology(ID:integer):string;
begin
  result:='';
  assignfile(f,FName);
  {$I-} reset(f); {$I+}
  if IOResult<>0 then exit;

  while not eof(f) do
  begin
    read(f,SyndRec);
    if SyndRec.ID=ID then
    begin
       result:=SyndRec.Pathology;
       break;
    end;
  end;
  closefile(f);
end;                    


Usually this seams to work fine, but for some records I get the "external: EXEC_BAD_ACCESS" error. Then I get the execution paused message with "CFStringGetLength", and the compiler is stopped at the following function in the CarbonEdits unit:

function TCarbonControlWithEdit.SetText(const S: String): Boolean;
var
  CFString: CFStringRef;
begin
  Result := False;
  if GetEditPart < 0 then Exit;
 
  CreateCFString(S, CFString);
  try
    if OSError(
      SetControlData(ControlRef(Widget), GetEditPart, kControlEditTextCFStringTag,
        SizeOf(CFStringRef), @CFString), // --> compiler breaks on this line
      Self, SSetText, SSetData) then Exit;
     
    Result := True;
  finally
    FreeCFString(CFString);
  end;
end;                                          


The data is read into the record but seems to fail when using the string it was converted to. I can't find any strange characters in these records. Maybe a comma ( ' / " ) might cause a bug on the CFString although it is accepted with Delphi on windows?

Any idea what it might be, and more important, how to solve it?

Thanks.

[Phil]

Any idea what it might be, and more important, how to solve it?

Thanks.

The code you show is non-GUI in nature, but the error you report involves the Carbon widgetset implementation of the LCL (GUI).

I assume Pathology is a null-terminated string of chars. If so, try using StrPas or casting it to a string or something before returning as a function result.

Thanks.

-Phil

[Jo]

I don't think the #0 is the issue. I've looped over the string and found no #0 problems. Also, the strpas is doing the following which is alredy in my function:

function StrPas(Str: PChar): string;
begin
  Result:=Str;
end ;

[Jo]

I've made several more tests. Loading the record and the creation of a string is ok and writing back to a file is ok too. The error appears when using the gui, inserting the string into a memo or using showmessage box etc.

I then checked each character in two records with error messages, and came up with the following examples.

In the first record, the quotation marks have been replaced (observe the characters surrounding the word "pattern"):

"This is a description of a certain character or personality, rather than a ìpatternî. It includes ...."


In the second record an apostrophe had been replaced by the same character (observe the word "person's"):

"However, in chronic cases, when the personís Qi has been weakened..."

(I hope the html will show the above text as-is and will not change the characters still into something else.)

I will appreciate any help. Thanks.

[Tombo]

Note that Carbon interface is expecting UTF_8 encoded strings - maybe you are using ANSI, then you should use AnsiToUtf8 for conversion.
If this is not your case, report it as a bug (see http://wiki.lazarus.freepascal.org/How_do_I_create_a_bug_report) with test application.

[Jo]

Thanks, but this is not a utf8 problem, seems it is my lack of knowledge. This is very basic, but since these are my first steps with the Mac I guess I’ve missed it.

The thing is that one cannot transfer text files between windows and Mac without some conversions routines, even though the text itself appears the same. These are the main issues:

7 bit ASCII text - need to change:  #13#10 into #13
8 bit ASCII text – need to change most of the >128 ASCII chars

A good page that discusses it and have the chars tables:
http://www1.tip.nl/~t876506/charsets.html

[Phil]

Thanks, but this is not a utf8 problem, seems it is my lack of knowledge. This is very basic, but since these are my first steps with the Mac I guess I’ve missed it.

The thing is that one cannot transfer text files between windows and Mac without some conversions routines, even though the text itself appears the same. These are the main issues:

7 bit ASCII text - need to change:  #13#10 into #13
8 bit ASCII text – need to change most of the >128 ASCII chars

A good page that discusses it and have the chars tables:
http://www1.tip.nl/~t876506/charsets.html

I believe Tombo is correct. You can't pass chars > 128 to most of the Lazarus widgetsets (including Carbon) without converting them first to UTF8. I have a program with data files from Windows that includes characters like copyright ($A9) and degree ($B0). I need to convert the text using AnsiToUTF8 before I can use it in Laz controls.

Don't use the conversion tables you reference. Carbon needs UTF8 (multi-byte chars if original char is > 128).

Thanks.

-Phil

[skalogryz]

Jo,
you may also use WideString, so any required convertion is done automatically.

Code: [Select]


procedure AddPathalogy(const rec: TSyndRec)
var
  s : WideString;
begin
  s := rec.Pathology;
  Memo1.Lines.Add(s);
end;


can you give a sample of array of char, that is cannot be added to a memo (or an edit?).
something like: $FF $FE $FE...

[Jo]

Tombo, Phil, you are right that AnsiToUTF8 helps to stop the error messages when passing to a widetset, but it will not make the correct translation of ASCII>128. I translated the original files again in windows, made few changes, for example, change char #147 to #34 (") and now everything works fine, both with the AnsiToUTF8 and without.  

skalogryyz , Just try this, and you'll get the error:
Memo1.Lines.Add(#147);

Thank you all.

[Phil]

Tombo, Phil, you are right that AnsiToUTF8 helps to stop the error messages when passing to a widetset, but it will not make the correct translation of ASCII>128. I translated the original files again in windows, made few changes, for example, change char #147 to #34 (") and now everything works fine, both with the AnsiToUTF8 and without.  

skalogryyz , Just try this, and you'll get the error:
Memo1.Lines.Add(#147);

Thank you all.

All you did was convert to lower (<128) ASCII it sounds like.

See what two chars AnsiToUTF8 converts #147 - I get $E2 $80. Look these up on the Mac Unicode char app and see if that isn't the curly open double quote.

Passing in a Widestring doesn't make sense to me. Most of the widgetsets expect UTF8 strings (normal 1-byte strings with special encoding).

Thanks.

-Phil

[Phil]

Tombo, Phil, you are right that AnsiToUTF8 helps to stop the error messages when passing to a widetset, but it will not make the correct translation of ASCII>128. I translated the original files again in windows, made few changes, for example, change char #147 to #34 (") and now everything works fine, both with the AnsiToUTF8 and without.  

skalogryyz , Just try this, and you'll get the error:
Memo1.Lines.Add(#147);

Thank you all.

Just to be sure, test Memo1.Lines.Add(WideString(#147));

It's easy to become confused by all the encodings, etc.

Thanks.

-Phil

[Phil]

Just to be sure, test Memo1.Lines.Add(WideString(#147));

It's easy to become confused by all the encodings, etc.

I see I've seriously confused myself by not thinking the whole thing through. On my Windows system, AnsiToUTF8 of #147 returns 3 chars: $E2 $80 $9C. Converting #147 to a WideString gives a single 2-byte char consisting of $20 $1C. However, I notice that WideString(#147) with FPC doesn't work the same as Delphi - not sure what that means.

Thanks.

-Phil

[skalogryz]

in Delphi, AnsiStrings and constants are converted to unicode by MultiByteToWideChar windows API function, using system default charset (depends upon system locale).

PS. For Mac OS i'm using WideStrings only, because CFStrings are UTF-16 based. It's faster to Move data from WideString to CFString buffer, rather than to convert from Utf8. Carbon also allows to create CFStrings that shares WideString memory, so no additional coping is required.
That's why I can't rely upon FPC/LCL routines, and have to use my own.