Lazarus

Free Pascal => Windows => Topic started by: dbannon on May 14, 2022, 09:06:06 am

Title: [solved]fcl-json and newlines
Post by: dbannon on May 14, 2022, 09:06:06 am
Using the FP json parser to read some json with a text field with newline characters surprised me on Windows.  I understand that correct JSON has an embedded "\n" in text where a newline is needed. And it works fine under Linux but does exactly the same thing under Windows, that is, converts "\n" to a LineFeed, not the Carriage Return, Line Feed that Windows needs. Thus -

Code: Pascal  [Select][+][-]
  1. uses fpjson, jsonparser;
  2. .....
  3. procedure TFormDump.Button1Click(Sender: TObject);
  4. var
  5.   jData : TJSONData;
  6.   jObject : TJSONObject;
  7.   s  : string;
  8.   AWord : String = '';
  9.   i : integer;
  10. begin
  11.    jData := GetJSON('{"Fld1" : "Hello\nAgain"}');
  12.    S := jObject.Get('Fld1');
  13.    Dump;
  14.    jData.Free;
  15.    Memo1.Clear;
  16.    for i := 1 to length(S) do         // report on content of S
  17.        if (S[i] > ' ') and (S[i] <= '~') then
  18.            AWord := AWord + S[i]
  19.        else begin
  20.            Memo1.Append(AWord);           // one word at a time.
  21.            AWord := '';
  22.            case ord(S[i]) of
  23.               10  : Memo1.Append('--------- Line Feed');
  24.               13  : Memo1.Append('--------- Carriage Ret');
  25.            else
  26.                Memo1.Append('--------- Ord ' + inttostr(ord(InString[i])));
  27.            end;
  28.        end;
  29.   if AWord <> '' then Memo1.Append(AWord);
  30. end;

Produces -

Code: [Select]
Hello
--------- Line Feed
Again

On both Linux and Windows. And, of course, Windows does not like it.

Just what am I missing here ?

Davo
Title: Re: fcl-json and newlines
Post by: six1 on May 14, 2022, 09:25:41 am
In case of Windows, try to replace #10 to #13 may be a solution
Code: Pascal  [Select][+][-]
  1. ...
  2.   begin
  3.      jData := GetJSON('{"Fld1" : "Hello\nAgain"}');
  4.      S := jObject.Get('Fld1');
  5.     {$IFDEF Windows}
  6.       s:=stringreplace(s,#10,#13,[rfreplaceall]);
  7.     {$endif}
  8. ...
  9.  
Title: Re: fcl-json and newlines
Post by: AlexTP on May 14, 2022, 09:29:36 am
fpJSON converts '\n' to LF chars, because '\n' means LF. If you need CR LF newlines, you need to use text with '\r\n', or you can also replace LF -> CR LF by StringReplace.
Title: Re: fcl-json and newlines
Post by: dbannon on May 14, 2022, 01:34:11 pm
Thanks both six1 and Alex but thats not really the point, while its easy to work around, I don't think it should be like that.

"\n" is, obviously, a C newline, migrated to Javascript and then to JSON, by time it get to there, its not a Linefeed, its a newline, should be translated to a pascal LineEnding, and thats either #10 on Unix, #13#10 on Windows. (was #13 on Mac long, long ago).

Or so I think .....

Davo
Title: Re: fcl-json and newlines
Post by: Thausand on May 14, 2022, 02:25:21 pm
"\n" is, obviously, a C newline, migrated to Javascript and then to JSON, by time it get to there, its not a Linefeed, its a newline, should be translated to a pascal LineEnding, and thats either #10 on Unix, #13#10 on Windows. (was #13 on Mac long, long ago).

Or so I think .....
Not really. here: https://www.json.org/json-en.html

Special escape for linefeed, carriage return etc.
Title: Re: fcl-json and newlines
Post by: Thaddy on May 14, 2022, 02:42:10 pm
And that link is the ONLY reputable JSON doc, btw... Everything else is bolt-on rubbish.
I wrote a grammar for the real standard that can output Pascal. Many implementations use that as core.
It is on Embarcadero.
Title: Re: fcl-json and newlines
Post by: dbannon on May 15, 2022, 02:12:18 am
> Not really. here: https://www.json.org/json-en.html

And nicely written too. Better than the resources I have been using.  But still not totally clean about newline IMHO. While it mentions LineFeed and CarriageReturn it also says -

The Unicode code point U+000A is used as the newline.

Embedding "\u000A" instead of "\n" produces the same result, just a LineFeed on Windows

Thaddy, your JSON parser, made for Windows (or Delphi), did it convert "newline" to #10 or #13#10 on Windows ?

Davo

EDIT: OK, I give up, JSON Tools does the same thing, so right or wrong, thats how it it is !
Code: Pascal  [Select][+][-]
  1. {$ifdef WINDOWS}St := St.replace('#10', '#13#10', [rfReplaceAll]);{$endif}
:D
Title: Re: fcl-json and newlines
Post by: Thausand on May 15, 2022, 03:30:41 am
While it mentions LineFeed and CarriageReturn it also says -
The Unicode code point U+000A is used as the newline.
Maybe you can help ? Because /me can not find it  :-[

"Newline" not exist for json.

ECMA document (https://www.ecma-international.org/wp-content/uploads/ECMA-404_2nd_edition_december_2017.pdf) writes:
Quote
9 String
A string is a sequence of Unicode code points wrapped with quotation marks (U+0022). All code points may be placed within the quotation marks except for the code points that must be escaped: quotation mark
(U+0022), reverse solidus (U+005C), and the control characters U+0000 to U+001F. There are two-character escape sequence representations of some characters.
\" represents the quotation mark character (U+0022).
\\ represents the reverse solidus character (U+005C).
\/ represents the solidus character (U+002F).
\b represents the backspace character (U+0008).
\f represents the form feed character (U+000C).
\n represents the line feed character (U+000A).
\r represents the carriage return character (U+000D).
\t represents the character tabulation character (U+0009).
So, for example, a string containing only a single reverse solidus character may be represented as "\\".

json \n (Line feed) <> c \n (newline).

Newline is afaik c-ishm and platform depending, see https://en.wikipedia.org/wiki/Newline#In_programming_languages

Quote
Embedding "\u000A" instead of "\n" produces the same result, just a LineFeed on Windows
Yes, see pdf ecma: it be linefeed.

I not know how to explain better (my English is bit better but not good). I think you wish for \n be JSON newline (I understand why) but that create same problem for JSON that is there for newline (example: TStringlist platform compatibility when read/write text lines). TStringlist solve problem but TStringlist is Pascal class and developer can make rules. JSON is official standard and developer have to follow (or fork/make new standard  :D )
Title: Re: fcl-json and newlines
Post by: dbannon on May 15, 2022, 09:57:23 am
Maybe you can help ? Because /me can not find it  :-[
"Newline" not exist for json.
From the link you posted, down right hand side, click on any of the entries under "escapes" and it takes you to https://www.crockford.com/mckeeman.html - not far down from the top.  But I suspect its not expressly about JS

Quote
...see https://en.wikipedia.org/wiki/Newline#In_programming_languages
Java, PHP, and Python provide the '\r\n' sequence (for ASCII CR+LF). In contrast to C, these are guaranteed to represent the values U+000D and U+000A, respectively.
I am convinced !  I always considered a Java \n pretty much the same as a C one and therefore JS also the same, but I was wrong !

Quote
I not know how to explain better (my English is bit better but not good).
You have made yourself quite understandable, far, far better than I would be in your native language !

Thanks
Title: Re: fcl-json and newlines
Post by: Thausand on May 15, 2022, 01:45:11 pm
From the link you posted, down right hand side, click on any of the entries under "escapes" and it takes you to https://www.crockford.com/mckeeman.html - not far down from the top.  But I suspect its not expressly about JS
Thank you, light is now on  :)

not expressly = not explicit ?

If yes, then in first line of link it write:
Quote
McKeeman Form
This is an excerpt from Chapter 22 of How JavaScript Works.
Later/down it write about JSON.

Quote
This is the JSON grammar in McKeeman Form.

Hope it helped.
TinyPortal © 2005-2018