Recent

Author Topic: client doesn't show data  (Read 8030 times)

mangakissa

  • Hero Member
  • *****
  • Posts: 1131
client doesn't show data
« on: June 29, 2020, 04:31:08 pm »
With this link I build a very basic restserver with time.
The server is located on localhost:9080. With my browser I get a result without problems.
Now I built with pas2js a simple page with one button and a dataset. If I run the restserver in debug mode I can see the clinet is doing a request. The response is sent back to the client. But the client is doing nothing.

I put an writeln in procedure DSOpen() but the writeln will not displaying in browser.
The doGetUrl() and btnTimeClick() are displaying well.

Why will my dataset not open and display data in my browser?
Lazarus 2.06 (64b) / FPC 3.0.4 / Windows 10
stucked on Delphi 10.3.1

ps

  • Full Member
  • ***
  • Posts: 136
    • CSS
Re: client doesn't show data
« Reply #1 on: June 29, 2020, 08:33:39 pm »
It's client and server on same domain?
Small simple CSS/box model implementation: https://github.com/pst2d/csscontrols/tree/dev

TRon

  • Hero Member
  • *****
  • Posts: 2432
Re: client doesn't show data
« Reply #2 on: June 30, 2020, 06:40:49 am »
Hi mangakissa,

Thank you for your example.

I am completely oblivious with regards to pas2js so I have no idea what I am doing (other then following wiki pages and forum posts).

I added some output messages to the server, then fired it up.

Then I loaded your html page, which loads your script generated by pas2js.

That showed me a button. When I pressed that the server showed:
Code: [Select]
2020.06.30 07:27:04:315 : Server: default routing from time
2020.06.30 07:27:04:317 : Server: Send json response
Which are my output messages.

The browser window (chromium) showed me:
Code: [Select]
button clicked
URL :http://localhost:9080/time
loaded.....

The browser console showed me:
Code: [Select]
button clicked
client.js:12458 URL :http://localhost:9080/time
client.js:12458 loaded.....
client.html:1 Access to XMLHttpRequest at 'http://localhost:9080/time' from origin 'null' has been blocked by CORS policy: No 'Access-Control-Allow-Origin' header is present on the requested resource.

And... that is as far as I was able to comprehend, other than the hint to add a header ?

PascalDragon

  • Hero Member
  • *****
  • Posts: 5446
  • Compiler Developer
Re: client doesn't show data
« Reply #3 on: June 30, 2020, 09:27:28 am »
The browser console showed me:
Code: [Select]
button clicked
client.js:12458 URL :http://localhost:9080/time
client.js:12458 loaded.....
client.html:1 Access to XMLHttpRequest at 'http://localhost:9080/time' from origin 'null' has been blocked by CORS policy: No 'Access-Control-Allow-Origin' header is present on the requested resource.

And... that is as far as I was able to comprehend, other than the hint to add a header ?

The missing CORS header is exactly the problem. You either need to set that header correctly (though I don't know right now how it needs to be set for a local file...) or you need to distribute the HTML file and JS script using the restserver as well. I'd suggest the later.

If you use FPC 3.2.0 you can do this rather easily like this:

Code: Pascal  [Select][+][-]
  1. // add unit fpWebFile to your uses clause of the server project
  2.  
  3. begin
  4.   Application.Port := 9080;
  5.   HTTPRouter.RegisterRoute('/time', @timeEndpoint, true);
  6.   HTTPRouter.RegisterRoute('/greeting/:name', @greetingEndpoint);
  7.   // here you need to use the bath to the client data relative to your executable
  8.   TSimpleFileModule.BaseDir := ExpandFileName('../client');
  9.   // you need a file with mime types, though I *think*
  10.   // that FPC 3.2.0 can also do this without
  11.   // you'll notice if it works or not if your browser tries to
  12.   // download the HTML file instead of displaying it ;)
  13.   MimeTypesFile := ExpandFileName('./mime.types');
  14.   // pick some route where you want your webserver to serve the
  15.   // files from
  16.   HTTPRouter.RegisterRoute('/client/*', @TSimpleFileModule.HandleSimpleFileRequest);
  17.   Application.Threaded := true;
  18.   Application.Initialize;
  19.   Application.Run;
  20. end.

TRon

  • Hero Member
  • *****
  • Posts: 2432
Re: client doesn't show data
« Reply #4 on: June 30, 2020, 11:31:55 am »
 Thank you PascalDragon.

Unfortunately that seems to complicate matters even more for me .... (do note that i'm easily confused :-) )

I have server and client both located in the same directory so i used:
Code: Pascal  [Select][+][-]
  1.  ...
  2.   TSimpleFileModule.Basedir := ExpandFileName('./client');
  3.   // mimetypes ...needed for 3.2 ?
  4.   MimeTypesFile := ExpandFileName('./mime.types');  
  5.   // pick some route where you want the server to serve the files from
  6.   HTTPRouter.RegisterRoute('./*', @TSimpleFileModule.HandleSimpleFileRequest);
  7. ...
  8.  
html/js response from client is identical as mentioned before, however the console does reply in a different manner:
Code: [Select]
client.js:12458 button clicked
client.js:12458 URL :http://localhost:9080/time
client.js:12458 loaded.....
client.js:12331 GET http://localhost:9080/time net::ERR_CONNECTION_REFUSED

The result from using another route for the filerequest (which i tried first, using previous pasted mimetype and client location):
Code: Text  [Select][+][-]
  1. ..
  2.   HTTPRouter.RegisterRoute('*', @TSimpleFileModule.HandleSimpleFileRequest);
  3. ..
  4.  
resulted in a console output of:
Code: [Select]
button clicked
client.js:12458 URL :http://localhost:9080/time
client.js:12458 loaded.....
client.html:1 Access to XMLHttpRequest at 'http://localhost:9080/time' from origin 'null' has been blocked by CORS policy: No 'Access-Control-Allow-Origin' header is present on the requested resource.
client.js:12331 Cross-Origin Read Blocking (CORB) blocked cross-origin response http://localhost:9080/time with MIME type application/json. See https://www.chromestatus.com/feature/5629709824032768 for more details.

I do not even have an idea of what is the correct routing there..... the last seems more promising, but at the same time seems more wrong to me....

I even tried "/*" but that seems to act in the same manner as the results in my first post. The only difference to that is that after so many repeated (click button) retries it adds the additional message "Cross-Origin Read Blocking (CORB) blocked cross-origin response http://localhost:9080/time with MIME type application/json. See https://www.chromestatus.com/feature/5629709824032768 for more details."

I did had a read about that CORB blocking, and an answer from SO seems to suggest installing an extension/plugin (yet another one) to circumvent this blocking issue, but i'm not that stupid as it should just simply be a matter of proper configuration.

mangakissa

  • Hero Member
  • *****
  • Posts: 1131
Re: client doesn't show data
« Reply #5 on: June 30, 2020, 02:25:35 pm »
Quote
Unfortunately that seems to complicate matters even more for me .... (do note that i'm easily confused :-) )
But I am.

It's all about streaming, isn't it?
The expanded code above does give met this message:
Quote
[Debugger Exception Notification]

Project project1 raised exception class 'EHTTP' with message:
No REQUEST_METHOD passed from server.

 At address 100027DC2


[Ignore this exception type]
I have disabled CORRS on firefox.

I'm using Lazarus 2.1.0 / FPC 3.20 on using pas2js
« Last Edit: June 30, 2020, 02:27:33 pm by mangakissa »
Lazarus 2.06 (64b) / FPC 3.0.4 / Windows 10
stucked on Delphi 10.3.1

PascalDragon

  • Hero Member
  • *****
  • Posts: 5446
  • Compiler Developer
Re: client doesn't show data
« Reply #6 on: June 30, 2020, 03:13:25 pm »
Thank you PascalDragon.

Unfortunately that seems to complicate matters even more for me .... (do note that i'm easily confused :-) )

I have server and client both located in the same directory so i used:
Code: Pascal  [Select][+][-]
  1.  ...
  2.   TSimpleFileModule.Basedir := ExpandFileName('./client');
  3.   // mimetypes ...needed for 3.2 ?
  4.   MimeTypesFile := ExpandFileName('./mime.types');  
  5.   // pick some route where you want the server to serve the files from
  6.   HTTPRouter.RegisterRoute('./*', @TSimpleFileModule.HandleSimpleFileRequest);
  7. ...
  8.  

I've now checked the provided example myself (the other code was from my own project). I have not tested to put the HTML and JS file into the same directory as the server however. The following works if the server directory is next to the client directory:

Code: Pascal  [Select][+][-]
  1. begin
  2.   Application.Port := 9080;
  3.   HTTPRouter.RegisterRoute('/time', @timeEndpoint, true);
  4.   HTTPRouter.RegisterRoute('/greeting/:name', @greetingEndpoint);
  5.   TSimpleFileModule.BaseDir := ExpandFileName('../');
  6.   MimeTypesFile := 'mime.types';
  7.   HTTPRouter.RegisterRoute('/client/*', @TSimpleFileModule.HandleSimpleFileRequest);
  8.   Application.Threaded := true;
  9.   Application.Initialize;
  10.   Application.Run;
  11. end.

There won't be a CORS error this way as long as you call the HTML site using localhost. If you want to publish this on a different computer (even if it's the same LAN) then you need to make sure that your Pas2JS code does not load from localhost, but maybe preferred using just the URI instead: /time instead of http://localhost/time. Alternatively you'll need to make sure that the server is set correctly (in what way ever).

The only error I now get is that the JSON DB parser can't find any fields.

TRon

  • Hero Member
  • *****
  • Posts: 2432
Re: client doesn't show data
« Reply #7 on: June 30, 2020, 10:08:26 pm »
Thank you PascalDragon.

Unfortunately i couldn't get it to work as seem intended. So, i ended up manipulating the response header manually.
Code: Pascal  [Select][+][-]
  1.   res.SetFieldByName('Access-Control-Allow-Origin', '*');
  2.  
which resulted in the browser with:

Code: [Select]
button clicked
URL :http://localhost:9080/time
loaded.....
DoAfterLoad
..and the console with:

Code: [Select]
button clicked
client.js:12458 URL :http://localhost:9080/time
client.js:12458 loaded.....
client.js:12458 DoAfterLoad
client.js:11797 Uncaught
details:
Code: [Select]
__proto__:
$ancestor: {$ancestor: {…}, $class: {…}, $classname: "EDatabaseError", $parent: {…}, $module: {…}, …}
$class: {$ancestor: {…}, $class: {…}, $classname: "EJSONDataset", $parent: {…}, $module: {…}, …}
$classname: "EJSONDataset"
$fullname: "JSONDataset.EJSONDataset"
$module: {$name: "JSONDataset", $intfuseslist: Array(8), $impluseslist: Array(1), $state: 5, $intfcode: ƒ, …}
$name: "EJSONDataset"
$parent: {$name: "JSONDataset", $intfuseslist: Array(8), $impluseslist: Array(1), $state: 5, $intfcode: ƒ, …}
$rtti: {name: "EJSONDataset", $module: {…}, class: {…}, members: {…}, names: Array(0), …}
__proto__: Object

With details on the uncaught
Code: [Select]
InternalInitFieldDefs @ client.js:11797
InternalOpen @ client.js:11746
InternalOpen @ client.js:12048
DoInternalOpen @ client.js:6481
OpenCursor @ client.js:6949
SetActive @ client.js:6982
Open @ client.js:7450
HandleRequestresponse @ client.js:6577
cb @ client.js:217
DoAfterRequest @ client.js:7785
onLoad @ client.js:12371
cb @ client.js:217
load (async)
DoGetData @ client.js:12315
DoLoad @ client.js:6601
Load @ client.js:7371
btnTimeClick @ client.js:12526
cb @ client.js:217

FPC 3.2.0, Raspbian Linux arm (in case that is useful for mangakissa)

PascalDragon

  • Hero Member
  • *****
  • Posts: 5446
  • Compiler Developer
Re: client doesn't show data
« Reply #8 on: July 01, 2020, 09:29:13 am »
Unfortunately i couldn't get it to work as seem intended. So, i ended up manipulating the response header manually.
Code: Pascal  [Select][+][-]
  1.   res.SetFieldByName('Access-Control-Allow-Origin', '*');
  2.  

You should really try to find out why it's necessary for you, cause setting a general CORS header (one with an asterisk instead of specific domains) is a bad idea (TM) - at least if one intends to publish the server on the web.

TRon

  • Hero Member
  • *****
  • Posts: 2432
Re: client doesn't show data
« Reply #9 on: July 01, 2020, 10:47:03 am »
You should really try to find out why it's necessary for you, cause setting a general CORS header (one with an asterisk instead of specific domains) is a bad idea (TM) - at least if one intends to publish the server on the web.
Sorry, I keep forgetting we're on a public forum....  :-[

Yes, i need to figure out the why and yes it is a bad idea.

Before I did resort to using the asterisk, I did read the answers from this SO question: https://stackoverflow.com/questions/10636611/how-does-access-control-allow-origin-header-work
Which explains how the header works, what you should use for this header (in case you do use it) and explains (or links to) the risks/pitfalls involved.

My main focus was to get at the point where user mangakissa apparently already is, in order to try to understand what is going wrong. I'm trying as hard as I possibly can to catch up on this whole js thing, and got a little stuck on the jsondataset. There seems to be no documentation ? and going through the sources goes slow because everything seems to go back'n'forth and forth'n'back again (multiple times)  :D

I do apologise in case i'm sort of derailing mangakissa's thread. Initially I had the impression he had/has similar issues that I encountered in this thread.

edit: judging by the restserver example the server from mangakissa does not appear to send (all) the required data that a jsondataset seem to expect, such as f.e. metadata header
« Last Edit: July 01, 2020, 11:52:33 am by TRon »

mangakissa

  • Hero Member
  • *****
  • Posts: 1131
Re: client doesn't show data
« Reply #10 on: July 01, 2020, 01:20:20 pm »
But in my example to asking time there's no metadata. It just calls a procedure to create an json response sent sent it back to client.

Why to I get data if I put 'http://localhost:9080/time' in my browser? Is it sending hidden data to call the restserver?
Lazarus 2.06 (64b) / FPC 3.0.4 / Windows 10
stucked on Delphi 10.3.1

TRon

  • Hero Member
  • *****
  • Posts: 2432
Re: client doesn't show data
« Reply #11 on: July 01, 2020, 01:30:25 pm »
But in my example to asking time there's no metadata. It just calls a procedure to create an json response sent sent it back to client.
Yes, but as far as i am able to tell that is exactly what is the issue.

You open a jsondataset (RestDataSet) on the resulting string. As such the client expects to receive the proper data (information).

Quote
Why to I get data if I put 'http://localhost:9080/time' in my browser? Is it sending hidden data to call the restserver?
For the dataset to work you need to follow a certain pattern, just as you would do for a normal dataset. A normal dataset (let's call it table) has filednames definitions and then rows of data that fill that database.

Using the TRestdataset requires something similar, at least that is what my tests are showing.

What i have now is that the server sends back:
Code: Text  [Select][+][-]
  1. {
  2.   "metaData" : {
  3.     "fields" : [
  4.       {
  5.         "name" : "time",
  6.         "type" : "string"
  7.       }
  8.     ],
  9.     "root" : "Data"
  10.   },
  11.   "Data" : [
  12.     {
  13.       "time" : "13:21:57"
  14.     }
  15.   ]
  16. }
  17.  
And the browser (client) responds to that with:
Code: [Select]
button clicked
URL :http://localhost:9080/time
loaded.....
DoAfterLoad
Dataset opened
1 -13:21:57
URL :http://localhost:9080/time
Which seems to be what you are attempting to accomplish ?

If that is not the case then we need to have a conversation on what it is that you are exactly expecting or are trying to accomplish :)

edit PS: do note that I have less then 24 hours experience with pas2js, so in case my writings are incorrect then please correct me.

edit2: and I have no idea why but it seems each time my server receives two GET requests (and responds to them as well)
« Last Edit: July 01, 2020, 01:43:48 pm by TRon »

mangakissa

  • Hero Member
  • *****
  • Posts: 1131
Re: client doesn't show data
« Reply #12 on: July 01, 2020, 02:56:45 pm »
Yes, that's correct. But this response must be created manually.
I did this in my other topic. But I got stuck with no response. So I looked to on the forum and found a link: how to create a simple restserver.

My actual project calls a complete list by pressing the search button. Build the testserver with the required (header) fields.
I see then coming as a log in my restserver, but when the response must sent back a streamwrite error occurs.

And I can't figure it out why.

(I looked for two weeks now, but al time together about one day all)
« Last Edit: July 01, 2020, 02:58:25 pm by mangakissa »
Lazarus 2.06 (64b) / FPC 3.0.4 / Windows 10
stucked on Delphi 10.3.1

TRon

  • Hero Member
  • *****
  • Posts: 2432
Re: client doesn't show data
« Reply #13 on: July 01, 2020, 03:31:56 pm »
I did this in my other topic. But I got stuck with no response.
Sorry, I have no idea you already started a topic about this... topic  :)

Quote
My actual project calls a complete list by pressing the search button. Build the testserver with the required (header) fields.
I see then coming as a log in my restserver, but when the response must sent back a streamwrite error occurs.

And I can't figure it out why.
I have no idea why (yet) but this is what my chromium browser returns in the console:
Code: [Select]
Uncaught
fMessage: "Field definition 9 in has no or invalid name property"
Also, not sure if it should happen before or after that exception but the search field wasn't mentioned by server, and i seems also to be missing the content of my search query.

Perhaps you have an idea about the fieldname ?. I will look through the code and see if I am able to track that down but I do not have many time left for today.

TRon

  • Hero Member
  • *****
  • Posts: 2432
Re: client doesn't show data
« Reply #14 on: July 01, 2020, 04:03:59 pm »
I have no idea why (yet) but this is what my chromium browser returns in the console:
Code: [Select]
Uncaught
fMessage: "Field definition 9 in has no or invalid name property"
Perhaps you have an idea about the fieldname ?. I will look through the code and see if I am able to track that down but I do not have many time left for today.
Got it, I think  :)

Inside restdata.pp, line 58 (here 11 highlighted):
Code: Pascal  [Select][+][-]
  1.     MetaData.Arrays['fields'].Add(TJSONObject.Create(
  2.     ['supplier',
  3.     'product',
  4.     'type',
  5.     'size',
  6.     'reference',
  7.     'package',
  8.     'quantity',
  9.      'pkquantity',
  10.      'location',
  11.      'int']));
  12.  
I was thrown off by the error message only mentioning "in', and not "int"

I have not been able to test further yet.

edit: ok, I can see you got your queue from the demo and I noticed on the first testruns (scroll down to highlighted):
Code: Text  [Select][+][-]
  1. {
  2.   "metaData" : {
  3.     "fields" : [
  4.       {
  5.         "name" : "supplier",
  6.         "type" : "string"
  7.       },
  8.       {
  9.         "name" : "product",
  10.         "type" : "string"
  11.       },
  12.       {
  13.         "name" : "type",
  14.         "type" : "string"
  15.       },
  16.       {
  17.         "name" : "size",
  18.         "type" : "string"
  19.       },
  20.       {
  21.         "name" : "reference",
  22.         "type" : "string"
  23.       },
  24.       {
  25.         "name" : "package",
  26.         "type" : "string"
  27.       },
  28.       {
  29.         "name" : "quantity",
  30.         "type" : "int"
  31.       },
  32.       {
  33.         "name" : "pkquantity",
  34.         "type" : "string"
  35.       },
  36.       {
  37.         "name" : "location",
  38.         "type" : "string"
  39.       },
  40.       {
  41.         "supplier" : "product",
  42.         "type" : "size",
  43.         "reference" : "package",
  44.         "quantity" : "pkquantity",
  45.         "location" : "int"
  46.       }
  47.     ],
  48.     "root" : "Data"
  49.   },
  50.   "Data" :
  51.  
Which currently doesn't make any sense to me. Why is it there, inside the JSON (metadata) and as a field-def ?

When I removed the int, I got an exception (missing pair value):
Code: [Select]
An unhandled exception occurred at $000D7558:
EJSON: TJSONObject must be constructed with name,value pairs
  $000D7558  TJSONDATA__DOERROR,  line 1382 of ./fcl-json/src/fpjson.pp
  $000DBF28  TJSONOBJECT__CREATE,  line 3200 of ./fcl-json/src/fpjson.pp
  $0007C0C4  LOADRESTDATA,  line 57 of restdata.pp
  $0007D0DC  RESTDATA_$$_init$,  line 239 of restdata.pp
  $00032E70  fpc_initializeunits,  line 969 of /home/Pierre/pas/release-build/release_3_2_0/fpcsrc/rtl/inc/system.in
which makes sense as fielddefs come in pairs, and those do not look like field-definitions to me. Unless I'm missing something that I am not aware off with regards to TRestDataSet, otherwise this seems to be your (current) issue.

edit2: in the restserver example, it seems that the original (stored on disk) data is extended (before sending to the client) with an extra field with the name "id" of type "int". That is why the restserver example seems to be 'fixing' all the data by adding an id field using the following loop:
Code: Pascal  [Select][+][-]
  1.     MetaData.Arrays['fields'].Add(TJSONObject.Create(['name','id','type','int']));
  2.     For I:=0 to Data.Count-1 do
  3.       Data.Objects[i].Add('id',I+1);
  4.     LastID:=Data.Count;
  5.  
.. which makes sense if you actually make use of such an id. However as you can see from your own code, you seem to be doing (or wanting) something else there ?
« Last Edit: July 01, 2020, 04:44:37 pm by TRon »

 

TinyPortal © 2005-2018