Recent

Author Topic: [SOLVED] REST client over HTTPS - where to start?  (Read 9370 times)

Mick

  • Jr. Member
  • **
  • Posts: 51
[SOLVED] REST client over HTTPS - where to start?
« on: September 25, 2018, 08:39:04 pm »
Hi,
At the moment I have some brainstorming on the technology to pick from for the development of a task.
The task to be done is to get some data from one REST endpoint and put it into another one (kind of a simple data-migration tool).
I'm considering cloud-based solution (Java-based application deployed in one of the cloud services).
But since the tool should have some user friendly GUI (to pick tables, records criteria etc.), I also consider Lazarus as an option to choose from.
The high-level idea is to implement a REST client that would connect via https to the REST source endpoint, collect the data locally and then push the data to REST destination endpoint (the pre-requisite is that both systems have the same REST API, only the endpoints are different).
What I also need is to be able to manipulate custom HTTP headers (authentication, etc.).
Both SOAP and JSON are the options for transferring the data.

Is Lazarus a good pick for such task?
Where to start with the POC development?
What package/library (with a not too steep learning curve) would be the best to use?
« Last Edit: September 27, 2018, 06:50:13 pm by Mick »

valdir.marcos

  • Hero Member
  • *****
  • Posts: 1106
Re: REST client over HTTPS - where to start?
« Reply #1 on: September 26, 2018, 01:27:41 am »
Hi,
At the moment I have some brainstorming on the technology to pick from for the development of a task.
The task to be done is to get some data from one REST endpoint and put it into another one (kind of a simple data-migration tool).
I'm considering cloud-based solution (Java-based application deployed in one of the cloud services).
But since the tool should have some user friendly GUI (to pick tables, records criteria etc.), I also consider Lazarus as an option to choose from.
The high-level idea is to implement a REST client that would connect via https to the REST source endpoint, collect the data locally and then push the data to REST destination endpoint (the pre-requisite is that both systems have the same REST API, only the endpoints are different).
What I also need is to be able to manipulate custom HTTP headers (authentication, etc.).
Both SOAP and JSON are the options for transferring the data.

Is Lazarus a good pick for such task?
Where to start with the POC development?
What package/library (with a not too steep learning curve) would be the best to use?
Have you read?

REST Dataware Componentes
https://forum.lazarus.freepascal.org/index.php/topic,38281.0.html

My New Project Lazarus REST/JSON Server/Client
https://forum.lazarus.freepascal.org/index.php/topic,36290.0.html

Datasnap like servers with FPC/Lazarus
https://forum.lazarus.freepascal.org/index.php/topic,42139.0.html

Noob and JSON
https://forum.lazarus.freepascal.org/index.php/topic,41809.0.html

Que tipo de projeto usar?
https://forum.lazarus.freepascal.org/index.php/topic,38812.0.html

Mick

  • Jr. Member
  • **
  • Posts: 51
Re: REST client over HTTPS - where to start?
« Reply #2 on: September 26, 2018, 05:02:18 am »
Hi valdir.marcos,

Thanks for the links, I've went through these threads you suggested and actually - there are even more questions than answers for me after reading them. Maybe I will try to explain more what is on my mind and what is my experience so far.

I have experience with REST, but not with Delphi (or Lazarus). I have couple of years of experience with Delphi, but it was a couple of years ago when I've moved to Java, and then went to some cloud platform development. As for Lazarus - I've made a mid-sized project in Lazarus couple of months ago, but it was not related to REST, or even not related to networking at all, just a plain desktop app. So I have never created an app that would do similar things in Delphi or Lazarus. Therefore, as you can see, I'm not a noob when it comes to REST, JSON etc, however, I've only implemented such code (both clients and servers) with technologies other than Lazarus/FPC.

What I need to have (and what I don't need):

1. I only need to have REST client (I don't need the server, at least that's what I think right now).
2. I want to avoid the steep learning curve that some of the suggested component packages have. Most preferably, I'd like to use the standard component/packages/units if possible.
3. I want to be able to connect to a REST endpoint (HTTPS support is required).
4. I want to be able to easily manage the HTTP headers (to pass credentials, session Ids, security tokens etc depending on the cloud service provider security settings).
5. I want to be able to pull the data, preferably in a bulk approach (it is supported by the REST service).
6. For all of the the above - I think can it can be even done via simple HTTP(S) request. But maybe there is some simple wrapper library?
7. I want to be able to deserialize JSON (or SOAP) content from HTTP REST response of the source endpoint. But it should be done with some kind of abstract approach, because there is no fixed data structure. This means I don't need any ORM, because I think that there is no option to persist the results from the source endpoint in any temporary (local or remote) database. I only need to push it into the twin REST destination endpoint. Most probably I would store the parts of the JSON/SOAP responses representing individual records (or record batches) in a text files, because I will upload them into the destination REST endpoint using the very same format.

I'm not very familiar with the "standard" units/components provided with FPC/Lazarus (I mean these related to networking, JSON, etc.).
I'm looking forward to some hints what to pick or where to look for being able to do the implementation of a simple proof of concept.
And, BTW, I don't speak Portuguese or Spanish. English only (and Polish as my first language).

valdir.marcos

  • Hero Member
  • *****
  • Posts: 1106
Re: REST client over HTTPS - where to start?
« Reply #3 on: September 26, 2018, 02:17:38 pm »
Hi valdir.marcos,

Thanks for the links, I've went through these threads you suggested and actually - there are even more questions than answers for me after reading them. Maybe I will try to explain more what is on my mind and what is my experience so far.

I have experience with REST, but not with Delphi (or Lazarus). I have couple of years of experience with Delphi, but it was a couple of years ago when I've moved to Java, and then went to some cloud platform development. As for Lazarus - I've made a mid-sized project in Lazarus couple of months ago, but it was not related to REST, or even not related to networking at all, just a plain desktop app. So I have never created an app that would do similar things in Delphi or Lazarus. Therefore, as you can see, I'm not a noob when it comes to REST, JSON etc, however, I've only implemented such code (both clients and servers) with technologies other than Lazarus/FPC.

What I need to have (and what I don't need):

1. I only need to have REST client (I don't need the server, at least that's what I think right now).
2. I want to avoid the steep learning curve that some of the suggested component packages have. Most preferably, I'd like to use the standard component/packages/units if possible.
3. I want to be able to connect to a REST endpoint (HTTPS support is required).
4. I want to be able to easily manage the HTTP headers (to pass credentials, session Ids, security tokens etc depending on the cloud service provider security settings).
5. I want to be able to pull the data, preferably in a bulk approach (it is supported by the REST service).
6. For all of the the above - I think can it can be even done via simple HTTP(S) request. But maybe there is some simple wrapper library?
7. I want to be able to deserialize JSON (or SOAP) content from HTTP REST response of the source endpoint. But it should be done with some kind of abstract approach, because there is no fixed data structure. This means I don't need any ORM, because I think that there is no option to persist the results from the source endpoint in any temporary (local or remote) database. I only need to push it into the twin REST destination endpoint. Most probably I would store the parts of the JSON/SOAP responses representing individual records (or record batches) in a text files, because I will upload them into the destination REST endpoint using the very same format.

I'm not very familiar with the "standard" units/components provided with FPC/Lazarus (I mean these related to networking, JSON, etc.).
I'm looking forward to some hints what to pick or where to look for being able to do the implementation of a simple proof of concept.
And, BTW, I don't speak Portuguese or Spanish. English only (and Polish as my first language).
I understand what you mean, but most of my need is dataware-centric.
Usually, I have PHP on server side, sometimes it's Java, rarely it's CGI.
If you still don't have experience on Lazarus to work on advanced topics, it's difficult to avoid the steep learning curve even when you use the standard component/packages/units of Free Pascal and Lazarus.
Maybe a WEBService strategy should be more suited for your case.

fcl-json
http://wiki.freepascal.org/fcl-json

A Simple JSON Parsing Example
http://lazplanet.blogspot.com/2014/09/a-simple-json-parsing-example.html

Web Service Toolkit
http://wiki.lazarus.freepascal.org/Web_Service_Toolkit

WEBService
https://forum.lazarus.freepascal.org/index.php/topic,37208.0.html

Leledumbo

  • Hero Member
  • *****
  • Posts: 8746
  • Programming + Glam Metal + Tae Kwon Do = Me
Re: REST client over HTTPS - where to start?
« Reply #4 on: September 27, 2018, 03:02:35 pm »
fphttpclient and fcl-json is all you need for barebone starting point, which you can wrap into a single function (receiving and returning TJSONObject for flexibility). Everything else depends on you. WST can help generating client side if your server has WSDL file for the implemented SOAP protocol. REST, however, doesn't have any counterpart as it's not really a protocol but rather an architecture.

Mick

  • Jr. Member
  • **
  • Posts: 51
Re: REST client over HTTPS - where to start?
« Reply #5 on: September 27, 2018, 06:49:08 pm »
Thank you both valdir.marcos and Leledumbo,

I have actually started playing yesterday with TFPHTTPClient and TJSONObject and I have some promising results.
For now the disadvantage is that I need to dive into "manual" handling of REST API calls.
But - on the other hand - this gives me more flexibility to handle multiple scenarios.
At the moment I'm in the middle of creating some helpers and wrappers to make the callouts easy.

As for the WST - First, I prefer JSON over SOAP.
And even if I'd like to pick SOAP, I think it's not the best solution for me, because the code generation is involved.
IMHO that it would be an overkill if there are some changes introduced into WSDL quite often (and that is the case).
If I would like to provide support for every API version, it could be a challenge to maintain the code.
But I will need to think about the API versions anyway, so maybe I will come back to that idea.

Thanks and regards

 

TinyPortal © 2005-2018