* * *

Author Topic: [Solved using Indy] Replicate CURL command with FPHTTPClient  (Read 2194 times)

lainz

  • Hero Member
  • *****
  • Posts: 2825
    • Home
Re: [Solved using Indy] Replicate CURL command with FPHTTPClient
« Reply #15 on: April 19, 2018, 10:34:39 pm »
FPHTTPCLIENT:
Code: Pascal  [Select]
  1. {
  2.   "args": {},
  3.   "data": "{ \"ConsultarVersion\" : {} }",
  4.   "files": {},
  5.   "form": {},
  6.   "headers": {
  7.     "Accept": "application/json",
  8.     "Connection": "close",
  9.     "Content-Length": "27",
  10.     "Content-Type": "application/json",
  11.     "Host": "httpbin.org"
  12.   },
  13.   "json": {
  14.     "ConsultarVersion": {}
  15.   },
  16.   "method": "POST",
  17.   "origin": "MY IP",
  18.   "url": "https://httpbin.org/anything"
  19. }

INDY:
Code: Pascal  [Select]
  1. {
  2.   "args": {},
  3.   "data": "{\"ConsultarVersion\":{}}",
  4.   "files": {},
  5.   "form": {},
  6.   "headers": {
  7.     "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8",
  8.     "Connection": "close",
  9.     "Content-Length": "23",
  10.     "Content-Type": "application/json",
  11.     "Host": "httpbin.org",
  12.     "User-Agent": "Mozilla/3.0 (compatible; Indy Library)"
  13.   },
  14.   "json": {
  15.     "ConsultarVersion": {}
  16.   },
  17.   "method": "POST",
  18.   "origin": "MY IP",
  19.   "url": "https://httpbin.org/anything"
  20. }
  21.  
  22.  

lainz

  • Hero Member
  • *****
  • Posts: 2825
    • Home
Re: [Solved using Indy] Replicate CURL command with FPHTTPClient
« Reply #16 on: April 20, 2018, 05:08:26 pm »
I noticed the differences between them, but also adding:

Code: Pascal  [Select]
  1. function TImpresoraFiscal.POST_FPHTTPClient(sJSON: string): string;
  2. var
  3.   client: TFPHTTPClient;
  4.   reqJSON: TStringStream;
  5. begin
  6.   ReqJson := TStringStream.Create(sJSON, TEncoding.UTF8);
  7.   client := TFPHttpClient.Create(nil);
  8.   try
  9.     client.AddHeader('Content-Type', 'application/json');
  10.     client.AddHeader('Accept',
  11.       'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8');
  12.     client.AddHeader('User-Agent', 'Mozilla/3.0 (compatible; Indy Library)');
  13.     client.RequestBody := reqJSON;
  14.     Result := client.Post(URL_IMPRESORA);
  15.   finally
  16.     reqJSON.Free;
  17.     client.Free;
  18.   end;
  19. end;

And getting the exact response from the test site, It does not works, an always returns an empty string with fphttpclient.

engkin

  • Hero Member
  • *****
  • Posts: 2095
Re: [Solved using Indy] Replicate CURL command with FPHTTPClient
« Reply #17 on: April 20, 2018, 06:35:29 pm »
@lainz, what you posted in Reply #15 does not show everything. AFAIK Indy uses HTTP version 1.0 for posts. Also, Indy handles empty response and turns it into:
Code: Pascal  [Select]
  1.           if Length(LLocalHTTP.Response.ResponseText) = 0 then begin
  2.             // Support for HTTP responses without status line and headers
  3.             LLocalHTTP.Response.ResponseText := 'HTTP/1.0 200 OK'; {do not localize}

A quick test.

Indy:
Quote
POST /fiscal.json HTTP/1.0
----------: ----------
Content-Type: application/json
Content-Length: 23
Host: somehost.com
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
User-Agent: Mozilla/3.0 (compatible; Indy Library)

{"ConsultarVersion":{}}

To get FPHTTPClient to use version 1.0:
Code: Pascal  [Select]
  1.     client.HTTPVersion := '1.0';

Simple severs might expect Content-Length header directly after Content-Type header. So add it to the headers yourself:
Code: Pascal  [Select]
  1.     client.AddHeader('Content-Type', 'application/json');
  2.     client.AddHeader('Content-Length', IntToStr(length(sJSON)));

My test gave:
Quote
POST /fiscal.json HTTP/1.0
Host: somehost.com
Content-Type: application/json
Content-Length: 27
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*.*;q=0.8
User-Agent: Mozilla/5.0 (compatible; fpweb)
----------: -----

{ "ConsultarVersion" : {} }

I still see a difference.

Edit:
The last difference I saw -marked with red dashes above by one of my apps-:
Indy keeps the connection alive:
Quote
Connection: keep-alive

While FPHTTPClient:
Quote
Connection: close
« Last Edit: April 20, 2018, 06:55:38 pm by engkin »

lainz

  • Hero Member
  • *****
  • Posts: 2825
    • Home
Re: [Solved using Indy] Replicate CURL command with FPHTTPClient
« Reply #18 on: April 20, 2018, 07:55:05 pm »
Adding your suggested lines, I get still empty strings.

If anyone want to test, here is the emulator (Windows)
http://grupohasar.com/wp-content/uploads/2017/02/IFH-2G-Distribucion-180209.zip

The emulator is in the folder 'IFH-2G-Distribucion-180209\HERRAMIENTAS\EMULADOR WINDOWS\Emulador1000-777-240-101-171117 Trio4\1000.exe' and you need to configure a serial port with a tool called 'com0com' (IFH-2G-Distribucion-180209\HERRAMIENTAS\VIRTUALIZADOR PUERTOS SERIE) for x64, is a signed tool.

Ports are COM30 and COM31 when the tool installs and executes (setupg.exe).

Is really strange, for me it's a bug in the emulator, because I've used fphttpclient always and works fine, but not with this, maybe in the real device works, who knows...

Remy Lebeau

  • Sr. Member
  • ****
  • Posts: 439
    • Lebeau Software
Re: [Solved using Indy] Replicate CURL command with FPHTTPClient
« Reply #19 on: April 20, 2018, 09:39:30 pm »
AFAIK Indy uses HTTP version 1.0 for posts.

By default, yes.  You can disable that behavior by enabling the hoKeepOrigProtocol flag in the TIdHTTP.HTTPOptions property, then Post() will respect whichever version you specify in the TIdHTTP.ProtocolVersion property.

Also, Indy handles empty response and turns it into:
Code: Pascal  [Select]
  1.           if Length(LLocalHTTP.Response.ResponseText) = 0 then begin
  2.             // Support for HTTP responses without status line and headers
  3.             LLocalHTTP.Response.ResponseText := 'HTTP/1.0 200 OK'; {do not localize}

That is to handle old HTTP 0.9 servers, which don't send status lines or headers in responses (they also don't technically support POST either, only GET).

The ResponseText should NEVER be empty when communicating with an HTTP 1.0+ server.

Simple severs might expect Content-Length header directly after Content-Type header.

No HTTP server should *require* that, as it is forbidden by the HTTP specification to *require* headers be in any particular order.

Code: Pascal  [Select]
  1.     client.AddHeader('Content-Length', IntToStr(length(sJSON)));

That will fail to work if the JSON contains any non-ASCII characters.  The 'Content-Length' is the number of *bytes* being sent, not the number of *characters*.  So, if the JSON is encoded as UTF-8 during transmission, the 'Content-Length' must be set to the number of UTF-8 bytes being sent.

Indy keeps the connection alive:
Quote
Connection: keep-alive

HTTP 1.1 defaults to keep-alive semantics, unless you specify otherwise.  TIdHTTP explicitly requests keep-alive semantics in HTTP 1.0 unless you specify otherwise (in the TIdHTTP.Request.Connection property).  Ultimately, it is the server that decides whether a keep-alive is actually used or not, and that is reflected in the server's response (in the TIdHTTP.Response.KeepAlive property).

Keep-alive or not has no effect whatsoever on how the server processes a POST request, though.  It only affects whether the socket connection is kept open or is closed after the response is sent.
« Last Edit: April 20, 2018, 09:42:21 pm by Remy Lebeau »
Remy Lebeau
Lebeau Software - Owner, Developer
Internet Direct (Indy) - Admin, Developer (Support forum)

lainz

  • Hero Member
  • *****
  • Posts: 2825
    • Home
Re: [Solved using Indy] Replicate CURL command with FPHTTPClient
« Reply #20 on: May 12, 2018, 09:50:50 pm »
Using wireshark we get this:

Using Indy
Code: Pascal  [Select]
  1. POST /fiscal.json HTTP/1.0
  2. Connection: keep-alive
  3. Content-Type: application/json
  4. Content-Length: 22
  5. Host: 127.0.0.1
  6. Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
  7. User-Agent: Mozilla/3.0 (compatible; Indy Library)
  8.  
  9. {"ConsultarEstado":{}}{
  10.         "ConsultarEstado":
  11.         {
  12.                 "Secuencia": "68",
  13.                 "Estado":
  14.                 {
  15.                         "Impresora":
  16.                         [
  17.                         ],
  18.                         "Fiscal":
  19.                         [
  20.                                 "MemoriaFiscalInicializada"
  21.                         ]
  22.                 },
  23.                 "EstadoAuxiliar":
  24.                 [
  25.                 ],
  26.                 "EstadoInterno": "EnJornadaFiscal",
  27.                 "ComprobanteEnCurso": "NoDocumento",
  28.                 "CodigoComprobante": "Tique",
  29.                 "NumeroUltimoComprobante": "61",
  30.                 "CantidadEmitidos": "3",
  31.                 "CantidadCancelados": "0"
  32.         }
  33. }

Using fphttpclient (yes, is fphttpclient, I've added the user-agent and accept to match indy, to see if that helps..)
Code: Pascal  [Select]
  1. POST /fiscal.json HTTP/1.0
  2. Host: 127.0.0.1
  3. Content-Type: application/json
  4. Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
  5. User-Agent: Mozilla/3.0 (compatible; Indy Library)
  6. Content-Length: 22
  7. Connection: close
  8.  
  9. {"ConsultarEstado":{}}HTTP/1.0 200 Output Follows
  10. Server: HSL HTTP Server
  11. Date: Sat, 12 May 2018 15:41:39 GMT
  12. Connection: close
  13. Access-Control-Allow-Origin: *
  14. Access-Control-Allow-Methods: OPTIONS, POST, GET
  15. Access-Control-Max-Age: 86400
  16. Content-Type: application/json; charset=UTF-8
  17.  
  18. {
  19.         "ConsultarEstado":
  20.         {
  21.                 "Secuencia": "61",
  22.                 "Estado":
  23.                 {
  24.                         "Impresora":
  25.                         [
  26.                         ],
  27.                         "Fiscal":
  28.                         [
  29.                                 "MemoriaFiscalInicializada"
  30.                         ]
  31.                 },
  32.                 "EstadoAuxiliar":
  33.                 [
  34.                 ],
  35.                 "EstadoInterno": "EnJornadaFiscal",
  36.                 "ComprobanteEnCurso": "NoDocumento",
  37.                 "CodigoComprobante": "Tique",
  38.                 "NumeroUltimoComprobante": "63",
  39.                 "CantidadEmitidos": "5",
  40.                 "CantidadCancelados": "0"
  41.         }
  42. }

You can see in both cases, the emulator sends a response. But with fphttpclient it adds this extra stuff:

Code: Pascal  [Select]
  1. HTTP/1.0 200 Output Follows
  2. Server: HSL HTTP Server
  3. Date: Sat, 12 May 2018 15:41:39 GMT
  4. Connection: close
  5. Access-Control-Allow-Origin: *
  6. Access-Control-Allow-Methods: OPTIONS, POST, GET
  7. Access-Control-Max-Age: 86400
  8. Content-Type: application/json; charset=UTF-8

Maybe because originally tested with KeepConnection set to false..

I've set KeepConnection to true as well, but no luck.

Code: Pascal  [Select]
  1. class function TImpresoraFiscal.POST_FPHTTPClient(sJSON:
  2.   string): string;
  3. var
  4.   client: TFPHTTPClient;
  5.   reqJSON: TStringStream;
  6. begin
  7.   ReqJson := TStringStream.Create(sJSON, TEncoding.UTF8);
  8.   client := TFPHttpClient.Create(nil);
  9.   client.KeepConnection := True;
  10.   try
  11.     client.AddHeader('Content-Type', 'application/json');
  12.     client.AddHeader('Accept',
  13.       'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8');
  14.     client.AddHeader('User-Agent', 'Mozilla/3.0 (compatible; Indy Library)');
  15.     client.AddHeader('Content-Length', IntToStr(length(sJSON)));
  16.     //client.HTTPVersion := '1.0';
  17.     client.RequestBody := reqJSON;
  18.     try
  19.       Result := client.Post('http://' + IP + '/fiscal.json');
  20.     except
  21.       on e: Exception do
  22.         if Result = '' then
  23.           Result := '{}';
  24.     end;
  25.     if Result = '' then
  26.       Result := '{}';
  27.   finally
  28.     reqJSON.Free;
  29.     client.Free;
  30.   end;
  31. end;  

Remy Lebeau

  • Sr. Member
  • ****
  • Posts: 439
    • Lebeau Software
Re: [Solved using Indy] Replicate CURL command with FPHTTPClient
« Reply #21 on: May 14, 2018, 07:47:52 pm »
Using wireshark we get this:
...
You can see in both cases, the emulator sends a response. But with fphttpclient it adds this extra stuff:

Code: Pascal  [Select]
  1. HTTP/1.0 200 Output Follows
  2. Server: HSL HTTP Server
  3. Date: Sat, 12 May 2018 15:41:39 GMT
  4. Connection: close
  5. Access-Control-Allow-Origin: *
  6. Access-Control-Allow-Methods: OPTIONS, POST, GET
  7. Access-Control-Max-Age: 86400
  8. Content-Type: application/json; charset=UTF-8

If you were really looking at the raw output from Wireshark, you should be seeing the same HTTP response from the server whether you use Indy or FPHTTPClient.  The fact that you are not seeing the HTTP response headers when using Indy makes me think that you are NOT looking at the Wireshark output correctly, if you are even using Wireshark at all.

And just to confirm, you can use Indy's TIdHTTP.Response.ResponseText, TIdHTTP.Response.ResponseCode, and TIdHTTP.Response.RawHeaders properties to verify that you are, in fact, getting those response values from the server, even if you don't see them in Wireshark.
Remy Lebeau
Lebeau Software - Owner, Developer
Internet Direct (Indy) - Admin, Developer (Support forum)

lainz

  • Hero Member
  • *****
  • Posts: 2825
    • Home
Re: [Solved using Indy] Replicate CURL command with FPHTTPClient
« Reply #22 on: May 14, 2018, 08:40:16 pm »
Yes. We used Wireshark to visualize the data
But to capture we used a third-party tool because Wireshark can not capture localhost.

But of course Indy is the one is working fine. I have no problems with it.

We did that to debug fphttpclient. We detect that the emulator sends the data for both Indy and fphttpclient. But fphttpclient don't see that response.

Edit: is winpcap that doesn't capture localhost and we used rawcap.
« Last Edit: May 14, 2018, 08:45:35 pm by lainz »

 

Recent

Get Lazarus at SourceForge.net. Fast, secure and Free Open Source software downloads Open Hub project report for Lazarus