LGenerics: support for JSON Type Definition

[avk]

JSON Typedef(aka RFC 8927) is an easy-to-learn, standardized way to define a schema for JSON data.

I'm glad to announce that LGenerics has recently added some support for:

• validating JSON documents against JTD schemas
• generating Pascal code from JTD schemas(PoC implementation)

More information can be found in the Free Pascal Wiki.

[KodeZwerg]

Can it be included into OPM?

[KodeZwerg]

Maybe it should also be mentioned where to get (LGenerics)

[AlexTP]

Wiki writes about download location: https://wiki.freepascal.org/LGenerics#Download

[avk]

Can it be included into OPM?

If this question was addressed to me, then I don't know, sorry.

[totya]

@avk

As I wrote, I'm intrested in JSON processing, so I use the Trunk version.

Questions:
- Your library can load JSON files with UTF8 coding with BOM or without BOM?
- Your library can save JSON files with UTF8 coding with BOM or without BOM (I'd like manage this, for example SaveUTFWithBOM:=true;)

- Your library can load JSON files with Window(CR/LF) or Unix (LF) line ending?
- Your library can save JSON files with Window(CR/LF) or Unix (LF) line ending (I'd like manage this, for example SaveLineEnding:=leLinux;)

(My used and modified library know it, but I'd like to try and use your library.)

Quest:
I'd like to customize the format settings (space count), for example:

{
  "SampleKey": "SampleValue"
}

(I sign space with "x")

1. DefaultSpaceCountBeforeKey:=2; (xx"SampleKey")
Certainly this is the base, subkey use multiplier, so space count is 2, 4, 6, 8 and so on
or for example DefaultSpaceCountBeforeKey:=3; then space count is 3, 6, 9 and so on.

2. DefaultSpaceCountAfterColon:=1; (Mean Space after ":") (:x"SampleValue"

Sorry my English, and thank you.

Edit 1.: Schema.LoadFromFile function does not work, if this file is UTF8 BOM. (Without BOM function works.)

[totya]

@avk

How can I check (Validate) this schema?

Code: Pascal  [Select][+][-]

1. {
2.     "properties": {
3.         {"type": "string"}, {"type": "string"}
4.         }
5. }

JSon file structure sample:

Code: Pascal  [Select][+][-]

1. {
2.   "Rec1": "abc",
3.   "Rec2": "bde",
4.   "Rec3": "fgh"
5. }
6.  

[avk]

Thanks for your interest, LGenerics should compile with FPC-3.2.2, at least it did the last time I checked.
BTW, it is unlikely that my English is better than yours.
As to your questions:

- Your library can load JSON files with UTF8 coding with BOM or without BOM?

Byte Order Mark is not part of the standard. But a small exception is made for Utf8BOM: when JSON is loaded, a Boolean parameter(usually named SkipBom) is passed to the parser in most cases; if this parameter is set to True, Utf8BOM (and only Utf8BOM) will be ignored. Any other BOM will be rejected.

- Your library can save JSON files with UTF8 coding with BOM or without BOM (I'd like manage this, for example SaveUTFWithBOM:=true;)

There is no way to save JSON with BOM.

- Your library can load JSON files with Window(CR/LF) or Unix (LF) line ending?

Yes.

- Your library can save JSON files with Window(CR/LF) or Unix (LF) line ending (I'd like manage this, for example SaveLineEnding:=leLinux;)

There is little point in saving spaces and line endings, since they are not part of the standard and are simply ignored in parsing. TJsonNode, TJsonWriter, etc. save JSON in the most compact form, without spaces and line endings.

About formatting - the TJsonNode.FormatJson() function has a parameter that sets the indent value, but the indent after the colon is always one space.

How can I check (Validate) this schema?

The scheme for your JSON should be like this:

Code: Javascript  [Select][+][-]

1. {
2.     "properties": {
3.         "Rec1": {"type": "string"},
4.         "Rec2": {"type": "string"},
5.         "Rec3": {"type": "string"}
6.     }
7. }
8.  

[totya]

As to your questions:

Hi, very thank you for all answers!

There is no way to save JSON with BOM.

There is little point in saving spaces and line endings, since they are not part of the standard and are simply ignored in parsing.

Perhaps standard, but many hundred files created by this way (UTF8 BOM, Unix LF) I guess for a good reason. The processing app is unknown for me. I think I need to save manually then (own code).

The scheme for your JSON should be like this:

No, because "Rec1" "Rec2" are all variables, similar of this:

"Value": "Value"

Edit.: These are translations: many different IDs with language values

This is the reason why I wrote this:
{"type": "string"}, {"type": "string"}
but it don't work.

[avk]

If the property names change from instance to instance, then this is a Map (well, or a Dictionary, if you like).
The schema for the Dictionary should be like this:

Code: Javascript  [Select][+][-]

1. {
2.     "values": {"type": "string"}
3. }
4.  

[totya]

If the property names change from instance to instance, then this is a Map (well, or a Dictionary, if you like).
The schema for the Dictionary should be like this:

Code: Javascript  [Select][+][-]

1. {
2.     "values": {"type": "string"}
3. }
4.  

Thanks, but the information not much for me, if JSON damaged, for example the last " missing, I got this:

Error: instance path = , schema path = /values

I found error on the wiki:

Code: Pascal  [Select][+][-]

1.  Person := TJsonNode.Create;
2.   Person.AsJson := S;

The Validate(yep!) function is used to validate against the schema; its main parameters are the loaded JSON sample,

No, because .AsJon is parser, and exception raised if the JSON not correct.

So, before validate, suggest a Load function:

Code: Pascal  [Select][+][-]

1. Person := TJsonNode.Create;
2. Person.LoadFromFile

[avk]

No, because .AsJon is parser, and exception raised if the JSON not correct.

Yes it is, because

its main parameters are the loaded JSON sample,

Checking JSON syntax is a matter for the parser.  JTD only deals with the structure of syntactically correct constructions. These are very different things, and it is advisable not to mix them up in one pile.

[totya]

Code: Pascal  [Select][+][-]

1.  Person.AsJson := S;

This mean loaded for me. But no problem, my English much worst than yours.

I tried your json implementation, because I'd like to try the json validate function, but u see, I can't get usable error informations for these "translator" structured jsons. Perhaps error line number, or original json line more information would be.

By the way, the code generator function is quite fantastic, close the A.I.

[avk]

Thank you.

... I can't get usable error informations for these "translator" structured jsons. Perhaps error line number, or original json line more information would be.

I deliberately discarded the idea of tracking the error position in JSONs some time ago. The reasoning is simple:
  - if you create JSONs manually, there are plenty of online validators/formatters available for this purpose;
  - if invalid JSONs are created by some application of yours, you should look for errors in the application, not in JSONs;
  - if you receive invalid JSONs over the network, they should be rejected as invalid input without explanation;

But if you absolutely need to track the error position, maybe this approach would work: as far as I remember, the Fcl-Json parser tracks the error position, and the lgMiscUtils unit has a pair of functions to convert TJsonData to TJsonNode and vice versa.

[totya]

I deliberately discarded the idea of tracking the error position in JSONs some time ago. The reasoning is simple:

Yes, but this is the "special" (or normal?) situation, these are translator files, the translators modify them these line by line, so structure damage is possible. Thanks for the tips!