Recent

Author Topic: Optimize JSON values extraction  (Read 7797 times)

tudi_x

  • Hero Member
  • *****
  • Posts: 532
Optimize JSON values extraction
« on: June 16, 2017, 11:12:07 am »
hi All,
i am trying to extract the key:value information from recurrent JSON files as per attached app.
please advise if the extraction code may be optimized as the below looks complicated (in the DoURL and DoContent functions):

Code: Pascal  [Select][+][-]
  1. v1 := TJSONObject(TJSONObject(TJSONObject(AData).Items[0]).Items[j]).Names[l];
  2. v2 := TJSONObject(TJSONObject(TJSONObject(AData).Items[0]).Items[j]).FindPath(TJSONObject(TJSONObject(TJSONObject(AData).Items[0]).Items[j]).Names[l]).AsString;  

the json looks like the below:
Quote
{"info": {
    "users": { 
        "user1@gmail.com":{
            "idle:": false, 
            "me": true, 
            "away": false, 
            "name": "david", 
            "idleTime:": 686, 
            "qubicle": "Lobby", 
            "favourite": false
        },
        "user2@gmail.com": { 
            "idle:": false,
            "away": false,
            "name": "john",
            "idleTime:": 686,
            "qubicle": "Lobby",
            "favourite": false
        },
        "user3@gmail.com": { 
            "name": "mary",
            "loggedIn": false, 
            "favourite": true
        }
    }
}}
Lazarus 2.0.2 64b on Debian LXDE 10

JD

  • Hero Member
  • *****
  • Posts: 1848
Re: Optimize JSON values extraction
« Reply #1 on: June 16, 2017, 11:29:03 am »
Why is users not a JSON array?
Windows - Lazarus 2.1/FPC 3.2 (built using fpcupdeluxe),
Linux Mint - Lazarus 2.1/FPC 3.2 (built using fpcupdeluxe)

mORMot; Zeos 8; SQLite, PostgreSQL & MariaDB; VirtualTreeView

tudi_x

  • Hero Member
  • *****
  • Posts: 532
Re: Optimize JSON values extraction
« Reply #2 on: June 16, 2017, 11:37:41 am »
these are the requirements i received.
this is how the json is sent by the rest api.

the json content passes the rfc 4627 validation as per
https://jsonformatter.curiousconcept.com/#


please advise more, should there be an issue wiht the json?
Lazarus 2.0.2 64b on Debian LXDE 10

Blestan

  • Sr. Member
  • ****
  • Posts: 461
Re: Optimize JSON values extraction
« Reply #3 on: June 16, 2017, 01:24:37 pm »
you can use my fpc-xon package ... json parser is fast and the internal structures are easy to understand and use

try fpc-xon on github
Speak postscript or die!
Translate to pdf and live!

tudi_x

  • Hero Member
  • *****
  • Posts: 532
Re: Optimize JSON values extraction
« Reply #4 on: June 16, 2017, 01:58:21 pm »
do you have an example of how i could parse a recursive JSON?
i do not see an unmarshaling example on github.

thank you
Lazarus 2.0.2 64b on Debian LXDE 10

Blestan

  • Sr. Member
  • ****
  • Posts: 461
Re: Optimize JSON values extraction
« Reply #5 on: June 16, 2017, 03:11:11 pm »
the test application just parse  json in XVAR instance and put it in a treeview ...

i do not understand what u mean by recursive ...

... after parsin you can just use xon list object as a map

for example
 writeln(X['users'][1]['idletime'].asInteger)
Speak postscript or die!
Translate to pdf and live!

wp

  • Hero Member
  • *****
  • Posts: 11855
Re: Optimize JSON values extraction
« Reply #6 on: June 16, 2017, 03:28:30 pm »
do you have an example of how i could parse a recursive JSON?

You can have a look how it is done in the JSONDatastore of tvplaint:https://sourceforge.net/p/lazarus-ccr/svn/HEAD/tree/components/tvplanit/source/vpjsonds.pas:
TVpJSONDatastore.ReadJSON reads an entire json file which here consists of "Resources" which, in turn, consist of arrays of "Events", "Contacts" and "Tasks". JSONToEvent, for example, extracts all information needed for an event from the corresponding json array element.

I am aware that this is not a recursive algorithm because the structure of the file is well-defined. But maybe you only want to know how to parse a json in general.

JD

  • Hero Member
  • *****
  • Posts: 1848
Re: Optimize JSON values extraction
« Reply #7 on: June 16, 2017, 04:01:46 pm »
do you have an example of how i could parse a recursive JSON?
i do not see an unmarshaling example on github.

thank you

Here is how I fill a listview using Free Pascal's JSON library. It will give you an idea about how to use JSON objects ( with {}) and JSON arrays (with []). The parser reads the string. The procedure then loops through the array and then copies the each object in the array to a line in the listview. Each value in a key/value pair will be a column on a line. It is not recursive but it works. I hope you find it useful for your purpose.

Code: Pascal  [Select][+][-]
  1. procedure JSONObjToListViewEx(AListVu: TListView; AJSONObj: {wide}string; AHasCheckBoxes: boolean = False);
  2. var
  3.   intGridRow, intNumberOfSQLQueryRows: integer;
  4.   //
  5.   AddItem: TListItem;
  6.   // JSON related variables
  7.   jsParser: TJSONParser;
  8.   jsArrayRow: TJSONArray;
  9.   jsObj: TJSONObject;
  10. begin
  11.   //
  12.   if AJSONObj = EmptyStr then
  13.     exit;
  14.   // create the json parser
  15.   jsParser    := TJSONParser.Create(AJSONObj);
  16.   //
  17.   try
  18.     // Parse the JSON data
  19.     jsArrayRow  := jsParser.Parse as TJSONArray;
  20.     //
  21.     try
  22.       // get the number of rows returned by the query in the JSON data object
  23.       intNumberOfSQLQueryRows := jsArrayRow.Count;
  24.       // Fill the listview from the JSON string
  25.       AListVu.Items.BeginUpdate;
  26.       AListVu.Clear;
  27.       //
  28.       for intGridRow := 0 to Pred(intNumberOfSQLQueryRows) do
  29.       begin
  30.         // assign the object to jsObj
  31.         jsObj := jsArrayRow.Objects[intGridRow];
  32.         //
  33.         try
  34.           //
  35.           AddItem := AListVu.Items.Add;
  36.           with AddItem do
  37.           begin
  38.             if AHasCheckboxes then
  39.             begin
  40.               Caption := Trim(jsObj.Items[1].AsString);
  41.               SubItems.Add(jsObj.Items[0].AsString);      // ID
  42.             end
  43.             else if not AHasCheckboxes then
  44.             begin
  45.               Caption := Trim(jsObj.Items[0].AsString);   // ID
  46.               SubItems.Add(jsObj.Items[1].AsString);
  47.             end;
  48.           end;
  49.         finally
  50.           //
  51.           jsObj.Clear;
  52.         end;
  53.       end;
  54.     finally
  55.       //
  56.       AListVu.Items.EndUpdate;
  57.       //
  58.       jsArrayRow.Free;
  59.     end;          // try...finally (jsParser := TJSONParser.Create(AJSString))
  60.   finally
  61.     // free the jsParser
  62.     FreeAndNil(jsParser);
  63.   end;
  64. end;
  65.  


JD
« Last Edit: June 16, 2017, 04:09:35 pm by JD »
Windows - Lazarus 2.1/FPC 3.2 (built using fpcupdeluxe),
Linux Mint - Lazarus 2.1/FPC 3.2 (built using fpcupdeluxe)

mORMot; Zeos 8; SQLite, PostgreSQL & MariaDB; VirtualTreeView

JD

  • Hero Member
  • *****
  • Posts: 1848
Re: Optimize JSON values extraction
« Reply #8 on: June 16, 2017, 04:40:36 pm »
You could also see this article on LazPlanet:

A Simple JSON parsing example
https://lazplanet.blogspot.fr/2014/09/a-simple-json-parsing-example.html
Windows - Lazarus 2.1/FPC 3.2 (built using fpcupdeluxe),
Linux Mint - Lazarus 2.1/FPC 3.2 (built using fpcupdeluxe)

mORMot; Zeos 8; SQLite, PostgreSQL & MariaDB; VirtualTreeView

 

TinyPortal © 2005-2018