Recent

Author Topic: LGenerics: support for JSON Type Definition  (Read 7849 times)

avk

  • Hero Member
  • *****
  • Posts: 735
LGenerics: support for JSON Type Definition
« on: February 09, 2023, 03:39:34 pm »
JSON Typedef(aka RFC 8927) is an easy-to-learn, standardized way to define a schema for JSON data.

I'm glad to announce that LGenerics has recently added some support for:
  • validating JSON documents against JTD schemas
  • generating Pascal code from JTD schemas(PoC implementation)
More information can be found in the Free Pascal Wiki.

KodeZwerg

  • Hero Member
  • *****
  • Posts: 1398
  • Fifty shades of code.
    • Delphi & FreePascal
Re: LGenerics: support for JSON Type Definition
« Reply #1 on: February 09, 2023, 04:00:49 pm »
Can it be included into OPM?
« Last Edit: Tomorrow at 31:76:97 by KodeZwerg »

KodeZwerg

  • Hero Member
  • *****
  • Posts: 1398
  • Fifty shades of code.
    • Delphi & FreePascal
Re: LGenerics: support for JSON Type Definition
« Reply #2 on: February 09, 2023, 04:02:38 pm »
Maybe it should also be mentioned where to get (LGenerics)
« Last Edit: Tomorrow at 31:76:97 by KodeZwerg »

AlexTP

  • Hero Member
  • *****
  • Posts: 2118
    • UVviewsoft
Re: LGenerics: support for JSON Type Definition
« Reply #3 on: February 09, 2023, 04:06:10 pm »
Wiki writes about download location: https://wiki.freepascal.org/LGenerics#Download

avk

  • Hero Member
  • *****
  • Posts: 735
Re: LGenerics: support for JSON Type Definition
« Reply #4 on: February 10, 2023, 05:48:43 am »
Can it be included into OPM?

If this question was addressed to me, then I don't know, sorry.

totya

  • Hero Member
  • *****
  • Posts: 719
Re: LGenerics: support for JSON Type Definition
« Reply #5 on: February 10, 2023, 11:03:12 am »
@avk

As I wrote, I'm intrested in JSON processing, so I use the Trunk version.

Questions:
- Your library can load JSON files with UTF8 coding with BOM or without BOM?
- Your library can save JSON files with UTF8 coding with BOM or without BOM (I'd like manage this, for example SaveUTFWithBOM:=true;)

- Your library can load JSON files with Window(CR/LF) or Unix (LF) line ending?
- Your library can save JSON files with Window(CR/LF) or Unix (LF) line ending (I'd like manage this, for example SaveLineEnding:=leLinux;)

(My used and modified library know it, but I'd like to try and use your library.)

Quest:
I'd like to customize the format settings (space count), for example:

{
  "SampleKey": "SampleValue"
}

(I sign space with "x")

1. DefaultSpaceCountBeforeKey:=2; (xx"SampleKey")
Certainly this is the base, subkey use multiplier, so space count is 2, 4, 6, 8 and so on
or for example DefaultSpaceCountBeforeKey:=3; then space count is 3, 6, 9 and so on.

2. DefaultSpaceCountAfterColon:=1; (Mean Space after ":") (:x"SampleValue"

Sorry my English, and thank you.

Edit 1.: Schema.LoadFromFile function does not work, if this file is UTF8 BOM. (Without BOM function works.)
« Last Edit: February 10, 2023, 02:16:46 pm by totya »

totya

  • Hero Member
  • *****
  • Posts: 719
Re: LGenerics: support for JSON Type Definition
« Reply #6 on: February 10, 2023, 03:43:17 pm »
@avk

How can I check (Validate) this schema?

Code: Pascal  [Select][+][-]
  1. {
  2.     "properties": {
  3.         {"type": "string"}, {"type": "string"}
  4.         }
  5. }

JSon file structure sample:

Code: Pascal  [Select][+][-]
  1. {
  2.   "Rec1": "abc",
  3.   "Rec2": "bde",
  4.   "Rec3": "fgh"
  5. }
  6.  

avk

  • Hero Member
  • *****
  • Posts: 735
Re: LGenerics: support for JSON Type Definition
« Reply #7 on: February 10, 2023, 04:00:39 pm »
Thanks for your interest, LGenerics should compile with FPC-3.2.2, at least it did the last time I checked.
BTW, it is unlikely that my English is better than yours.
As to your questions:
Quote
- Your library can load JSON files with UTF8 coding with BOM or without BOM?

Byte Order Mark is not part of the standard. But a small exception is made for Utf8BOM: when JSON is loaded, a Boolean parameter(usually named SkipBom) is passed to the parser in most cases; if this parameter is set to True, Utf8BOM (and only Utf8BOM) will be ignored. Any other BOM will be rejected.
Quote
- Your library can save JSON files with UTF8 coding with BOM or without BOM (I'd like manage this, for example SaveUTFWithBOM:=true;)

There is no way to save JSON with BOM.
Quote
- Your library can load JSON files with Window(CR/LF) or Unix (LF) line ending?

Yes.
Quote
- Your library can save JSON files with Window(CR/LF) or Unix (LF) line ending (I'd like manage this, for example SaveLineEnding:=leLinux;)

There is little point in saving spaces and line endings, since they are not part of the standard and are simply ignored in parsing. TJsonNode, TJsonWriter, etc. save JSON in the most compact form, without spaces and line endings.

About formatting - the TJsonNode.FormatJson() function has a parameter that sets the indent value, but the indent after the colon is always one space.
Quote
How can I check (Validate) this schema?

The scheme for your JSON should be like this:
Code: Javascript  [Select][+][-]
  1. {
  2.     "properties": {
  3.         "Rec1": {"type": "string"},
  4.         "Rec2": {"type": "string"},
  5.         "Rec3": {"type": "string"}
  6.     }
  7. }
  8.  

totya

  • Hero Member
  • *****
  • Posts: 719
Re: LGenerics: support for JSON Type Definition
« Reply #8 on: February 10, 2023, 05:23:51 pm »
As to your questions:

Hi, very thank you for all answers!

Quote
There is no way to save JSON with BOM.
Quote
There is little point in saving spaces and line endings, since they are not part of the standard and are simply ignored in parsing.

Perhaps standard, but many hundred files created by this way (UTF8 BOM, Unix LF) I guess for a good reason. The processing app is unknown for me. I think I need to save manually then (own code).

Quote
The scheme for your JSON should be like this:

No, because "Rec1" "Rec2" are all variables, similar of this:

"Value": "Value"

Edit.: These are translations: many different IDs with language values

This is the reason why I wrote this:
{"type": "string"}, {"type": "string"}
but it don't work.
« Last Edit: February 10, 2023, 05:37:01 pm by totya »

avk

  • Hero Member
  • *****
  • Posts: 735
Re: LGenerics: support for JSON Type Definition
« Reply #9 on: February 10, 2023, 05:48:36 pm »
If the property names change from instance to instance, then this is a Map (well, or a Dictionary, if you like).
The schema for the Dictionary should be like this:
Code: Javascript  [Select][+][-]
  1. {
  2.     "values": {"type": "string"}
  3. }
  4.  

totya

  • Hero Member
  • *****
  • Posts: 719
Re: LGenerics: support for JSON Type Definition
« Reply #10 on: February 10, 2023, 06:30:15 pm »
If the property names change from instance to instance, then this is a Map (well, or a Dictionary, if you like).
The schema for the Dictionary should be like this:
Code: Javascript  [Select][+][-]
  1. {
  2.     "values": {"type": "string"}
  3. }
  4.  

Thanks, but the information not much for me, if JSON damaged, for example the last " missing, I got this:

Quote
Error: instance path = , schema path = /values


I found error on the wiki:

Code: Pascal  [Select][+][-]
  1.  Person := TJsonNode.Create;
  2.   Person.AsJson := S;

Quote
The Validate(yep!) function is used to validate against the schema; its main parameters are the loaded JSON sample,

No, because .AsJon is parser, and exception raised if the JSON not correct.

So, before validate, suggest a Load function:
Code: Pascal  [Select][+][-]
  1. Person := TJsonNode.Create;
  2. Person.LoadFromFile

avk

  • Hero Member
  • *****
  • Posts: 735
Re: LGenerics: support for JSON Type Definition
« Reply #11 on: February 10, 2023, 06:55:29 pm »
Quote
No, because .AsJon is parser, and exception raised if the JSON not correct.
Yes it is, because
Quote
its main parameters are the loaded JSON sample,
Checking JSON syntax is a matter for the parser.  JTD only deals with the structure of syntactically correct constructions. These are very different things, and it is advisable not to mix them up in one pile.


totya

  • Hero Member
  • *****
  • Posts: 719
Re: LGenerics: support for JSON Type Definition
« Reply #12 on: February 10, 2023, 07:18:49 pm »
Code: Pascal  [Select][+][-]
  1.  Person.AsJson := S;

This mean loaded for me. :) But no problem, my English much worst than yours.

I tried your json implementation, because I'd like to try the json validate function, but u see, I can't get usable error informations for these "translator" structured jsons. Perhaps error line number, or original json line more information would be.

By the way, the code generator function is quite fantastic, close the A.I. :)
« Last Edit: February 10, 2023, 07:22:04 pm by totya »

avk

  • Hero Member
  • *****
  • Posts: 735
Re: LGenerics: support for JSON Type Definition
« Reply #13 on: February 10, 2023, 07:55:35 pm »
Thank you.
Quote
... I can't get usable error informations for these "translator" structured jsons. Perhaps error line number, or original json line more information would be.

I deliberately discarded the idea of tracking the error position in JSONs some time ago. The reasoning is simple:
  - if you create JSONs manually, there are plenty of online validators/formatters available for this purpose;
  - if invalid JSONs are created by some application of yours, you should look for errors in the application, not in JSONs;
  - if you receive invalid JSONs over the network, they should be rejected as invalid input without explanation;

But if you absolutely need to track the error position, maybe this approach would work: as far as I remember, the Fcl-Json parser tracks the error position, and the lgMiscUtils unit has a pair of functions to convert TJsonData to TJsonNode and vice versa.

totya

  • Hero Member
  • *****
  • Posts: 719
Re: LGenerics: support for JSON Type Definition
« Reply #14 on: February 10, 2023, 07:59:37 pm »
I deliberately discarded the idea of tracking the error position in JSONs some time ago. The reasoning is simple:

Yes, but this is the "special" (or normal?) situation, these are translator files, the translators modify them these line by line, so structure damage is possible. Thanks for the tips!

 

TinyPortal © 2005-2018