Recent

Author Topic: Translation from Python to Lazarus  (Read 8676 times)

mpknap

  • Full Member
  • ***
  • Posts: 155
Translation from Python to Lazarus
« on: July 18, 2018, 07:24:26 am »
I have a request. The following code sends the file to the server. The main project is in Python, I do not know this programming environment. I do not know JSON. Anyone can write this in Lazarus Pascal for windows?



Code: Pascal  [Select][+][-]
  1. import_ requests
  2. import_ json
  3. import_ time
  4. import_ base64
  5. import_ urllib3
  6.  
  7. urllib3.disable_warnings()
  8. picture = open('test4.png', 'rb') # read picture
  9. picture_read = picture.read()
  10. base64_picture = base64.b64encode(picture_read) # save picture as  base 64
  11. print(len((str(base64_picture)[2:]))) # cutting off the first two characters from byte encoding, you may not need to
  12. linux_time = time.time()
  13. linux_time_millisecund = [item for item in str(linux_time) if item != '.'] # formatting time according to the linux era
  14.  
  15. # Below you have two variables in the form of a dictionary, which are protocols for logging in and sending data, this is what I sent you
  16. earlier and comes from the creed page (with minor changes)
  17.  
  18. # data in the login in addition to the password and e-mail are free, if you registered a cell phone it will match.
  19.  
  20. login = {'app_version': '1', 'device_model': 'zero', 'device_type': 'raspberry','system_version': 'raspbian',
  21. 'device_id': 'raspberry','password': 'hasło', "email": "email"}
  22.  
  23.  
  24. # in detection the data in addition to frame content and timestamp are also arbitrary, remember to limit the linux time to 13 characters as on the server,
  25. otherwise you will have problems with the date.
  26.  
  27. detection = {"detections": [{"frame_content": '%s'% (str(base64_picture)[2:]), "timestamp": int(''.join(linux_time_millisecund)[:-4]), "latitude": 51.708491,
  28. "longitude": 19.476362, "altitude": 12, "accuracy": 1, "provider": 'google map', "width": 45, "height": 45, "id": 1}],
  29. "device_id": "raspberry", "androidVersion": 'none', "device_model": "zero", "app_version": '1',
  30. "system_version": 'raspbian', "device_type": 'raspberry'}
  31.  
  32. # Once I've created variables, I save them in the json format.
  33.  
  34. with open("login.json", "w") as login_file:
  35.    json.dump(login, login_file)
  36. with open("login.json", "r") as read_login:
  37.    data_login = json.load(read_login)
  38.  
  39. with open("detection.json", "w") as detection_file:
  40.    json.dump(detection, detection_file)
  41. with open("detection.json", "r") as read_detection:
  42.    data_detetion = json.load(read_detection)
  43.  
  44. #request login is to get the authentication token needed to send the image to the server
  45.  
  46. request_login = requests.post('https://api.credo.science/api/v2/user/login', verify=False, json=data_login)
  47. print(request_login.status_code, request_login.reason)
  48. token = request_login.json()['token']
  49.  
  50. #in request detection you are sending a json file and a token code, and that's all, the file is on the server
  51.  
  52. request_detection = requests.post('https://api.credo.science/api/v2/detection', verify=False,
  53.                                   json=data_detetion, headers={'Authorization': 'Token %s' % token})
  54. print(request_detection.status_code, request_detection.reason)
  55. print(request_detection.content)
  56.  

sash

  • Sr. Member
  • ****
  • Posts: 366
Re: Translation from Python to Lazarus
« Reply #1 on: July 18, 2018, 02:05:44 pm »
I do not know this programming environment. I do not know JSON. Anyone can write this ...

Shouldn't it be posted in "jobs" then?
Lazarus 2.0.10 FPC 3.2.0 x86_64-linux-gtk2 @ Ubuntu 20.04 XFCE

mpknap

  • Full Member
  • ***
  • Posts: 155
Re: Translation from Python to Lazarus
« Reply #2 on: July 19, 2018, 08:59:42 pm »
it's not a job offer, it's a free project ;) I do not expect a ready solution (but it would be nice), I'm asking for tips, advice, whatever. I do not know how to start and whether it is possible to translate at all ...

sash

  • Sr. Member
  • ****
  • Posts: 366
Re: Translation from Python to Lazarus
« Reply #3 on: July 19, 2018, 10:14:36 pm »
Quote
I do not know how to start

Well, then learn basics about JSON, HTTP APIs, Base64 encoding, then basic Python syntax, and you'll be able to translate it for yourself.

I don't know Python either, but given code has comments, and as I can see, it uses some HTTP API (and API provider's address - so you can check their documentation).

Maybe you won't ever need a Python knowledge, if you know exact task, with their protocol description, you will be able to recreate Pascal code from scratch, with some tweaking and debugging.
Lazarus 2.0.10 FPC 3.2.0 x86_64-linux-gtk2 @ Ubuntu 20.04 XFCE

engkin

  • Hero Member
  • *****
  • Posts: 3112
Re: Translation from Python to Lazarus
« Reply #4 on: July 20, 2018, 06:22:53 am »
To read the picture:
Code: Pascal  [Select][+][-]
  1. uses Classes...
  2.  
  3. var
  4.   picture: TFileStream;
  5.   picture_read: array of byte;
  6. ...
  7.   picture := TFileStream.Create('test4.png',fmOpenRead or fmShareDenyNone);
  8.   try
  9.     SetLength(picture_read, picture.Size);
  10.     picture.Read(picture_read[1], Length(picture_read));
  11.   finally
  12.     picture.Free;
  13.   end;

To convert the read picture to base64:
Code: Pascal  [Select][+][-]
  1. uses base64...
  2.  
  3. var
  4.   b64: TBase64EncodingStream;
  5.   base64_picture: TStringStream;
  6.   picture_read: array of byte;
  7. ...
  8.   base64_picture := TStringStream.Create('');
  9.   try
  10.     b64 := TBase64EncodingStream.Create(base64_picture);
  11.     try
  12.       b64.write(picture_read[1], Length(picture_read));
  13.     finally
  14.       b64.Free;
  15.     end;
  16.  
  17. //Do whatever with the result
  18.     WriteLn(base64_picture.DataString);
  19. //*************
  20.  
  21.   finally
  22.     base64_picture.Free;
  23.   end;

To generate JSON in accordance with their specs:
Code: Pascal  [Select][+][-]
  1. uses fpjson, ....
  2. var
  3.   O : TJSONObject;
  4.  
  5.   O := TJSONObject.Create([
  6.        'body',
  7.        TJSONObject.Create([
  8.        'detection',
  9.        TJSONObject.Create([
  10.                            'accuracy',1.0,
  11.                            'altitude',1.0,
  12.                            'frame_content',base64_picture.DataString
  13.  
  14.                              //add the rest here//}
  15.  
  16.                            ]),//detection
  17.  
  18.        'device_info',
  19.        TJSONObject.Create([
  20.                            'adnroidVersion','7.0',
  21.                            'deviceId','device Id',
  22.                            'deviceModel','device model'
  23.                            ]),//device_info
  24.  
  25.        'user_info',
  26.        TJSONObject.Create([
  27.                            'email','user@email.com',
  28.                            'key','access_key'
  29.                            ])//user_info
  30.  
  31.                          ]),//body
  32.        'header',
  33.        TJSONObject.Create([
  34.                            'application','1.3',
  35.                            'frame_type','detection',
  36.                            'protocol','1.0',
  37.                            'time_stamp',1518707872
  38.                            ])//header
  39.  
  40.                         ]);//root
  41.  
  42.   //See the result
  43.   Writeln(O.FormatJSon);
  44.   //**********

There is a better way to generate the JSON using classes. Take a look at Laz1.8.2\fpc\3.0.4\source\packages\fcl-json\examples specifically demortti.pp

mpknap

  • Full Member
  • ***
  • Posts: 155
Re: Translation from Python to Lazarus
« Reply #5 on: July 20, 2018, 08:15:54 am »
Thanks ENGKIN, it looks like it works. There is the last thing I have a problem with from the beginning. How to send it.
Code: Pascal  [Select][+][-]
  1. request_login = requests.post('https://api.credo.science/api/v2/user/login', verify=False, json=data_login)
  2. print(request_login.status_code, request_login.reason)
  3. token = request_login.json()['token']
  4.  
  5.  
  6.  
  7. request_detection = requests.post('https://api.credo.science/api/v2/detection', verify=False,
  8.                                   json=data_detetion, headers={'Authorization': 'Token %s' % token})
  9.  

Trenatos

  • Hero Member
  • *****
  • Posts: 535
    • MarcusFernstrom.com
Re: Translation from Python to Lazarus
« Reply #6 on: July 20, 2018, 05:12:45 pm »
Take a look at fphttp simple post.

engkin

  • Hero Member
  • *****
  • Posts: 3112
Re: Translation from Python to Lazarus
« Reply #7 on: July 21, 2018, 04:13:52 am »
There is the last thing I have a problem with from the beginning. How to send it.

To login/send your data use fphttpclient. Here is logging in:
Code: Pascal  [Select][+][-]
  1. uses fphttpcleint,...
  2. ...
  3. var
  4.   requests: TFPHTTPClient;
  5. ...
  6.   requests := TFPHTTPClient.Create(nil);
  7.   requests.AllowRedirect:=True;
  8.   try
  9.     data := TStringStream.Create('');
  10.     try
  11.       O.DumpJSON(data);  //<--- O/data holds whatever you want to send
  12.       requests.AddHeader('User-Agent','Your prefered agent');
  13.       requests.AddHeader('Content-type','application/json');
  14.       requests.RequestBody := data;
  15.       WriteLn('Response: ',requests.Post('https://api.credo.science/api/v2/user/login')); //<-- extract the token for here
  16.       WriteLn('Code: ', requests.ResponseStatusCode, 'Text: ',requests.ResponseStatusText);
  17.       WriteLn();
  18.     finally
  19.       data.Free;
  20.     end;
  21.   finally
  22.     requests.Free;
  23.   end;

To use secured connection "HTTPS" you need to get OpenSSL library. Have the files available to your application (same folder/add them to the correct system folder/add their path)

mpknap

  • Full Member
  • ***
  • Posts: 155
Re: Translation from Python to Lazarus
« Reply #8 on: July 26, 2018, 08:28:45 am »
It does not quite work for me. I did this procedure

Code: Pascal  [Select][+][-]
  1. procedure TForm1.Button3Click(Sender: TObject);
  2. var
  3.   requests: TFPHTTPClient;
  4.   data: TStringStream;
  5.  
  6. begin
  7.   requests := TFPHTTPClient.Create(nil);
  8.   requests.AllowRedirect:=True;
  9.   try
  10.     data := TStringStream.Create('');
  11.  
  12.   finally
  13.   end;
  14.   try
  15.     o := TJSONObject.Create([
  16.        'login',
  17.        TJSONObject.Create([
  18.                         'app_version',1,
  19.                         'device_model', 1,
  20.                        'device_type',1,
  21.                        'system_version',1,
  22.                          'device_id',1,
  23.                         'password', 'S73071YT6',
  24.                         'email', 'mpknap@wp.pl'
  25.                              ])
  26.                    ]);
  27.  
  28.  
  29.  
  30.        try
  31.       o.DumpJSON(data);  //<--- O/data holds whatever you want to send
  32.       requests.AddHeader('User-Agent','Your prefered agent');
  33.       requests.AddHeader('Content-type','application/json');
  34.       requests.RequestBody := data;
  35.      LABEL1.caption:=('Response: '+requests.Post('https://api.credo.science/api/v2/user/login')); //<-- extract the token for here
  36.       memo3.caption:=('Code: '+ inttostr(requests.ResponseStatusCode)+ 'Text: '+ (requests.ResponseStatusText));
  37.      finally
  38.       data.Free;
  39.     end;
  40.   finally
  41.     requests.Free;
  42.   end;
  43.  

After Break (Attached Image), the line from LABEL1 is underlined ...

engkin

  • Hero Member
  • *****
  • Posts: 3112
Re: Translation from Python to Lazarus
« Reply #9 on: July 26, 2018, 05:36:44 pm »
Your JSON is wrong. According to their data it should be:
Code: Pascal  [Select][+][-]
  1. {"app_version":1,"device_model":1,"device_type":1,"system_version":1,"device_id":1,"password":"S73071YT6","email":"mpknap@wp.pl"}

You can do that either directly:
Code: Pascal  [Select][+][-]
  1.     data := TStringStream.Create('{"app_version":1,"device_model":1,"device_type":1,"system_version":1,"device_id":1,"password":"S73071YT6","email":"mpknap@wp.pl"}');

or:
Code: Pascal  [Select][+][-]
  1.       o := TJSONObject.Create([
  2.                           'app_version',1,
  3.                           'device_model', 1,
  4.                          'device_type',1,
  5.                          'system_version',1,
  6.                            'device_id',1,
  7.                           'password', 'S73071YT6',
  8.                           'email', 'mpknap@wp.pl'
  9.                      ]);
  10.       O.DumpJSON(data);

I just tried it and the response was:
Quote
Response: {"username":"mpknap","display_name":"Marek .K","language":"pl","token":"S73071YT6","team":"credo Forum","email":"mpknap@wp.pl"}

Edit:
The code I used in my test:
Code: Pascal  [Select][+][-]
  1.   requests := TFPHTTPClient.Create(nil);
  2.   requests.AllowRedirect:=True;
  3.   try
  4.     //https://github.com/credo-science/credo-webapp/blob/master/credoapiv2
  5.     data := TStringStream.Create('{"app_version":1,"device_model":1,"device_type":1,"system_version":1,"device_id":1,"password":"S73071YT6","email":"mpknap@wp.pl"}');
  6.     try
  7.       requests.AddHeader('User-Agent','YourPreferedAgent');
  8.       requests.AddHeader('Content-type','application/json');
  9.       requests.RequestBody := data;
  10.       WriteLn('Response: ',requests.Post('https://api.credo.science/api/v2/user/login'));
  11.       WriteLn('Code: ', requests.ResponseStatusCode, ', Text: ',requests.ResponseStatusText);
  12.       WriteLn();
  13.     finally
  14.       data.Free;
  15.     end;
  16.   finally
  17.     requests.Free;
  18.   end;
« Last Edit: July 26, 2018, 05:40:47 pm by engkin »

mpknap

  • Full Member
  • ***
  • Posts: 155
Re: Translation from Python to Lazarus
« Reply #10 on: July 28, 2018, 09:08:59 am »
Dear Engkin I will not hesitate well. Json is not a subject for me, however. Since you have already helped a lot, I am asking you for the last step. We have received the Token. How to send the content of Detection. I am doing something wrong because I get the answer" Invalid authorization header". Please help and close the topic I will be grateful :)

Code: Pascal  [Select][+][-]
  1.  data := TStringStream.Create('{"app_version":1,"device_model":1,"device_type":1,"system_version":1,"device_id":1,"password":"S73071YT6","email":"mpknap@wp.pl"}');
  2.     detection := TStringStream.Create( '{"detections": [{"frame_content": "iVBORw0KGgoAAAANSUhEUgAAADIAAAAyCAAAAAA7VNdtAAACv0lEQVRIDVXBMVbrMBRAwfue5O8YQxYEHUVKejg07L+HJBLyv40KZuIZVbSiE9rRhhaUqKGG4hlVtKIT2tGGFpSooYbiGVW0ohPa0YYWlKihhuIZVbSiE9rRhhaUqKGG4hlVtKIT2tGGFpSooYbiBe3ohDb0gCoqqKAf1FG8oB2d0IYeUEUFFfSDOooXtKMT2tADqqiggn5QR/GCdnRCG3pAFRVU0A/qKF7Qjk5oQw+oooIK+kEdxSt6QhtaUUWJkqmjhuIVPaENraiiRMnUUUPxip7QhlZUUaJk6qiheEVPaEMrqihRMnXUULyiJ7ShFVWUKJk6aigu6Iw2tPDHggYaqKG4oDPa0MIfCxpooIbigs5oQwt/LGiggRqKCzqjDS38saCBBmooLuiMNrTwx4IGGqihuKAz2lFBgQJV1NEvGigu6Ix2VFCgQBV19IsGigs6ox0VFChQRR39ooHigs5oRwUFClRRR79ooLigM9pRQYECVdTRLxoo3tAj+ocKKiiZOmroG8UbekT/UEEFJVNHDX2jeEOP6B8qqKBk6qihbxRv6BH9QwUVlEwdNfSN4g09on+ooIKSqaOGvlF8oIoqCpQoUaA7uqIbig9UUUWBEiUKdEdXdEPxgSqqKFCiRIHu6IpuKD5QRRUFSpQo0B1d0Q3FB6qookCJEgW6oyu6oXhHK0oUKFCggRq6o47iHa0oUaBAgQZq6I46ine0okSBAgUaqKE76ije0YoSBQoUaKCG7qijeEcrShQoUKCBGrqjjuITJUqUKNGBBhrohjqKT5QoUaJEBxpooBvqKD5RokSJEh1ooIFuqKP4RIkSJUp0oIEGuqGO4hMlSpQo0YEGGuiGOoovpgMVpsHU0BUdKL6YDlSYBlNDV3Sg+GI6UGEaTA1d0YHii+lAhWkwNXRFB4ovpgMVpsHU0BUd6D8pRLGV1zBGtwAAAABJRU5ErkJggg==", "timestamp": "1528481412762", "latitude": "51.708491", "longitude": "19.476362", "altitude": "12", "accuracy": "1", "provider": "google map", "width": "680", "height": "460", "id": "1"}], "device_id": "raspberry", "androidVersion": "none", "device_model": "zero", "app_version": "1", "system_version": "raspbian", "device_type": "raspberry"}');
  3.  

engkin

  • Hero Member
  • *****
  • Posts: 3112
Re: Translation from Python to Lazarus
« Reply #11 on: July 28, 2018, 05:36:51 pm »
No problem at all. I'll be glad to help finish it as soon as I get a chance. Most likely tomorrow.

engkin

  • Hero Member
  • *****
  • Posts: 3112
Re: Translation from Python to Lazarus
« Reply #12 on: July 29, 2018, 07:53:30 am »
Get the token:
Code: Pascal  [Select][+][-]
  1. var
  2.   token: string;
  3. ...
  4.   try
  5.     //https://github.com/credo-science/credo-webapp/blob/master/credoapiv2
  6.     data := TStringStream.Create('{"app_version":1,"device_model":1,"device_type":1,"system_version":1,"device_id":1,"password":"S73071YT6","email":"mpknap@wp.pl"}');
  7.     try
  8.       requests.AddHeader('User-Agent','YourPreferedAgent');
  9.       requests.AddHeader('Content-type','application/json');
  10.       requests.RequestBody := data;
  11.       response := requests.Post('https://api.credo.science/api/v2/user/login');
  12.       WriteLn('Response: ',response);
  13.       WriteLn('Code: ', requests.ResponseStatusCode, ', Text: ',requests.ResponseStatusText);
  14.  
  15.       O := TJSONObject(GetJSON(response));
  16.       try
  17.         token := O.Get('token');
  18.       finally
  19.         O.Free;
  20.       end;
  21.  
  22.       WriteLn('Token: ',token);
  23.     finally
  24.       data.Free;
  25.     end;
  26.   finally
  27.     requests.Free;
  28.   end;

According to the docs, pass the token in the header:
Code: Pascal  [Select][+][-]
  1.   requests := TFPHTTPClient.Create(nil);
  2.   requests.AllowRedirect:=True;
  3.   try
  4.     detection:=TStringStream.Create('{"detections": [{"frame_content": "iVBORw0KGgoAAAANSUhEUgAAADIAAAAyCAAAAAA7VNdtAAACv0lEQVRIDVXBMVbrMBRAwfue5O8YQxYEHUVKejg07L+HJBLyv40KZuIZVbSiE9rRhhaUqKGG4hlVtKIT2tGGFpSooYbiGVW0ohPa0YYWlKihhuIZVbSiE9rRhhaUqKGG4hlVtKIT2tGGFpSooYbiBe3ohDb0gCoqqKAf1FG8oB2d0IYeUEUFFfSDOooXtKMT2tADqqiggn5QR/GCdnRCG3pAFRVU0A/qKF7Qjk5oQw+oooIK+kEdxSt6QhtaUUWJkqmjhuIVPaENraiiRMnUUUPxip7QhlZUUaJk6qiheEVPaEMrqihRMnXUULyiJ7ShFVWUKJk6aigu6Iw2tPDHggYaqKG4oDPa0MIfCxpooIbigs5oQwt/LGiggRqKCzqjDS38saCBBmooLuiMNrTwx4IGGqihuKAz2lFBgQJV1NEvGigu6Ix2VFCgQBV19IsGigs6ox0VFChQRR39ooHigs5oRwUFClRRR79ooLigM9pRQYECVdTRLxoo3tAj+ocKKiiZOmroG8UbekT/UEEFJVNHDX2jeEOP6B8qqKBk6qihbxRv6BH9QwUVlEwdNfSN4g09on+ooIKSqaOGvlF8oIoqCpQoUaA7uqIbig9UUUWBEiUKdEdXdEPxgSqqKFCiRIHu6IpuKD5QRRUFSpQo0B1d0Q3FB6qookCJEgW6oyu6oXhHK0oUKFCggRq6o47iHa0oUaBAgQZq6I46ine0okSBAgUaqKE76ije0YoSBQoUaKCG7qijeEcrShQoUKCBGrqjjuITJUqUKNGBBhrohjqKT5QoUaJEBxpooBvqKD5RokSJEh1ooIFuqKP4RIkSJUp0oIEGuqGO4hMlSpQo0YEGGuiGOoovpgMVpsHU0BUdKL6YDlSYBlNDV3Sg+GI6UGEaTA1d0YHii+lAhWkwNXRFB4ovpgMVpsHU0BUd6D8pRLGV1zBGtwAAAABJRU5ErkJggg==", "timestamp": "1528481412762", "latitude": "51.708491", "longitude": "19.476362", "altitude": "12", "accuracy": "1", "provider": "google map", "width": "680", "height": "460", "id": "1"}], "device_id": "raspberry", "androidVersion": "none", "device_model": "zero", "app_version": "1", "system_version": "raspbian", "device_type": "raspberry"}');
  5.     try
  6.       requests.AddHeader('User-Agent','YourPreferedAgent');
  7.       requests.AddHeader('Content-type','application/json');
  8.       requests.AddHeader('Authorization','Token '+token);
  9.       requests.RequestBody := detection;
  10.       WriteLn('Response: ',requests.Post('https://api.credo.science/api/v2/detection'));
  11.       WriteLn('Code: ', requests.ResponseStatusCode, ', Text: ',requests.ResponseStatusText);
  12.       WriteLn();
  13.     finally
  14.       detection.Free;
  15.     end;
  16.  
  17.   finally
  18.     requests.Free;
  19.   end;

The previous code resulted in:
Quote
Response: {"username":"mpknap","display_name":"Marek .K","language":"pl","token":"S73071YT6","team":"credo Forum","email":"mpknap@wp.pl"}
Code: 200, Text: OK
Token: S73071YT6
Response: {"detections":[{"id":1194285}]}
Code: 200, Text: OK

mpknap

  • Full Member
  • ***
  • Posts: 155
Re: Translation from Python to Lazarus
« Reply #13 on: July 29, 2018, 04:18:43 pm »
Looking at your code in the last answer, I see that I was close ;).

 Everything works, no errors, but why is not the upload done? It should show a new detection on

https://api.credo.science/web/user/mpknap/


In any case, thank you for everything, I would never solve this problem myself.

engkin

  • Hero Member
  • *****
  • Posts: 3112
Re: Translation from Python to Lazarus
« Reply #14 on: July 29, 2018, 06:05:08 pm »
Yes, you were almost there, and I am glad to help.

why is not the upload done? It should show a new detection on

https://api.credo.science/web/user/mpknap/
I don't know precisely, but I can try to guess:
  • The list is sorted based on [json reported] date. The one I did is 1528481412762ms ~ 2018.06.08
  • Maybe they filter detections that seem wrong. Like mismatching width/height compared to image details. (Actual: 50x50. Reported: 680x460)
  • Possibly you had uploaded that picture before?

You can download your detections using /api/v2/data_export to see if the data is there. Maybe it is time to get in touch with them and see if they have a better answer.

 

TinyPortal © 2005-2018