Recent

Author Topic: HTTP request over Unix Domain Socket  (Read 7573 times)

dubst3pp4

  • Jr. Member
  • **
  • Posts: 86
  • Retro computing ~ GNU/Linux
    • me on Mastodon
HTTP request over Unix Domain Socket
« on: December 02, 2017, 09:55:51 am »
Hello, I'm wondering if it is possible to send HTTP requests over the Unix Domain Socket? I want to query a service, which exposes an API via HTTP and Unix Domain Sockets, but I could not find out how to do this with TFPHTTPClient class... Any ideas?

Best regards,
Marc
Jabber: xmpp:marc.hanisch@member.fsf.org -- Support the Free Software Foundation: https://my.fsf.org/donate

Leledumbo

  • Hero Member
  • *****
  • Posts: 8747
  • Programming + Glam Metal + Tae Kwon Do = Me
Re: HTTP request over Unix Domain Socket
« Reply #1 on: December 02, 2017, 05:49:03 pm »
It requires additional support to connect over unix domain socket, which I don't think TFPHTTPClient has.

KemBill

  • Jr. Member
  • **
  • Posts: 74
Re: HTTP request over Unix Domain Socket
« Reply #2 on: December 02, 2017, 08:16:46 pm »
I know "simple sockets", but if the only difference between UNIX and IP

Code: Pascal  [Select][+][-]
  1. server_sockaddr.sun_family = AF_UNIX;  

instead

Code: Pascal  [Select][+][-]
  1. server_sockaddr.sun_family = AF_INET;  

you can try to overload the method

dubst3pp4

  • Jr. Member
  • **
  • Posts: 86
  • Retro computing ~ GNU/Linux
    • me on Mastodon
Re: HTTP request over Unix Domain Socket
« Reply #3 on: December 03, 2017, 05:47:09 pm »
It seems that the sockets unit is all what I need to connect to a Unix socket. Maybe subclassing the TFPHTTPClient class to use a Unix socket instead of an Internet socket should work... All the base HTTP functionality is already in the class, so it would be great if I could use it...
Jabber: xmpp:marc.hanisch@member.fsf.org -- Support the Free Software Foundation: https://my.fsf.org/donate

KemBill

  • Jr. Member
  • **
  • Posts: 74
Re: HTTP request over Unix Domain Socket
« Reply #4 on: December 03, 2017, 06:16:59 pm »
Are you sur you have nothing listening using netstat 127.0.0.1:80 ?

It seems you can inject a TSocketHandler class into TFPHTTPClient, so you just need to write a wrapper using TUnixSocket

dubst3pp4

  • Jr. Member
  • **
  • Posts: 86
  • Retro computing ~ GNU/Linux
    • me on Mastodon
Re: HTTP request over Unix Domain Socket
« Reply #5 on: December 04, 2017, 11:56:44 am »
Are you sur you have nothing listening using netstat 127.0.0.1:80 ?

It seems you can inject a TSocketHandler class into TFPHTTPClient, so you just need to write a wrapper using TUnixSocket
I'm trying to query the Docker daemon, which could be exposed to a real network host, but I want to avoid that.

I've played around for some time and connecting to the daemon is as easy as:

Code: Pascal  [Select][+][-]
  1. UnixSocket := TUnixSocket.Create('/var/run/docker.sock');

After that you can send the HTTP-Request with the WriteBuffer method and read data with the ReadBuffer method.

Now the interesting part:
the TFPHTTPClient class creates a TInetSocket instance for the private field FSocket, which is also of type TInetSocket, in the protected method ConnectToServer. There are no dependencies to the functionality of TInetSocket in any other methods, everywhere else just methods of the parent class TSocketStream of TInetSocket and TUnixSocket are used.

So if I could change FSocket from a private to a protected field and override the ConnectToServer method in a subclass, all should be fine. Unfortunately to change the private field in TFPHTTPClient, I would have to create a copy of the fphttpclient unit, wouldn't I? Any chance to take another route to solve this problem?

I also think it would be a good idea to contact the author of the unit (Michael Van Canneyt?) to propose such changes, so that the class is usable with Unix domain sockets, too (as more and more Linux services use HTTP over UDS, for example Docker and Node.js).
« Last Edit: December 04, 2017, 11:58:29 am by dubst3pp4 »
Jabber: xmpp:marc.hanisch@member.fsf.org -- Support the Free Software Foundation: https://my.fsf.org/donate

dubst3pp4

  • Jr. Member
  • **
  • Posts: 86
  • Retro computing ~ GNU/Linux
    • me on Mastodon
Re: HTTP request over Unix Domain Socket
« Reply #6 on: December 04, 2017, 03:54:30 pm »
I got it now working by changing the type of the FSocket field to FSocketStream. In my subclass I'm extending TFPHttpCustomClient by reimplementing the ConnectToServer method, which in turn creates an instance of TUnixSocket, while the original ConnectToServer creates a TInetSocket instance.

Maybe this is not the best solution, but it seems to work without problems. I would like to propose this change but don't know how or to whom...  :-\
Jabber: xmpp:marc.hanisch@member.fsf.org -- Support the Free Software Foundation: https://my.fsf.org/donate

Leledumbo

  • Hero Member
  • *****
  • Posts: 8747
  • Programming + Glam Metal + Tae Kwon Do = Me
Re: HTTP request over Unix Domain Socket
« Reply #7 on: December 04, 2017, 08:53:56 pm »
Maybe this is not the best solution, but it seems to work without problems. I would like to propose this change but don't know how or to whom...  :-\
Ask Michael, you already knew his name before.

dubst3pp4

  • Jr. Member
  • **
  • Posts: 86
  • Retro computing ~ GNU/Linux
    • me on Mastodon
Re: HTTP request over Unix Domain Socket
« Reply #8 on: December 05, 2017, 05:57:37 am »
Okay, I will send him an email  :)
Jabber: xmpp:marc.hanisch@member.fsf.org -- Support the Free Software Foundation: https://my.fsf.org/donate

istoica

  • New Member
  • *
  • Posts: 21
Re: HTTP request over Unix Domain Socket
« Reply #9 on: January 25, 2022, 11:10:24 am »
Have you managed to complete this ?

MarkMLl

  • Hero Member
  • *****
  • Posts: 6676
Re: HTTP request over Unix Domain Socket
« Reply #10 on: January 25, 2022, 11:15:16 am »
Have you managed to complete this ?

Necroposting will get you in trouble these days.

What are you trying to do: specifically use HTTP or use unix-domain sockets in general? If the latter then see this recent thread https://forum.lazarus.freepascal.org/index.php/topic,57706.msg429320.html#msg429320

MarkMLl
MT+86 & Turbo Pascal v1 on CCP/M-86, multitasking with LAN & graphics in 128Kb.
Pet hate: people who boast about the size and sophistication of their computer.
GitHub repositories: https://github.com/MarkMLl?tab=repositories

istoica

  • New Member
  • *
  • Posts: 21
Re: HTTP request over Unix Domain Socket
« Reply #11 on: January 25, 2022, 12:04:29 pm »
Thank you MarkMLI, I try to do both actually, exactly like the author of this thread.

Make an HTTP request over unix domain sockets, there are APIs such as docker api or podman api that due to security reasons, they don't expose http servers over TCP

https://docs.podman.io/en/latest/_static/api.html

curl --unix-socket /run/podman/podman.sock http://d/v3.0.0/libpod/info

- The curl client has built-in support for it these days
- A famous JS http client library axios, has `socketPath` as argument to support it - https://axios-http.com/docs/req_config
- Unix sockets are very useful for these scenarios too, even Windows supports it from 2017 - https://devblogs.microsoft.com/commandline/af_unix-comes-to-windows


So back to original, I simply want to use the simplicity of `TFPHTTPClient` using unix domain sockets. Tried to follow discoveries here, but arrived nowhere, segfault :)


Code: Pascal  [Select][+][-]
  1. unit Unit2;
  2.  
  3. {$mode ObjFPC}{$H+}
  4.  
  5. interface
  6.  
  7. uses
  8.   Classes, SysUtils, fphttpclient, ssockets, httpdefs, uriparser, strutils;
  9.  
  10. type
  11.  
  12.   TFPExHTTPClient = Class(TFPCustomHTTPClient)
  13.   private
  14.     FUnixSocketPath: string;
  15.     FSocket : TSocketStream;
  16.     FIOTimeout: Integer;
  17.     FConnectTimeout: Integer;
  18.   Public
  19.     Property UnixSocketPath: string Read FUnixSocketPath Write FUnixSocketPath;
  20.     Procedure ConnectToServer(const AHost: String; APort: Integer; UseSSL : Boolean=False); override;
  21.   end;
  22.  
  23. implementation
  24.    
  25.  
  26. procedure TFPExHTTPClient.ConnectToServer(const AHost: String;
  27.   APort: Integer; UseSSL : Boolean = False);
  28.  
  29. Var
  30.   G : TSocketHandler;
  31.  
  32.  
  33. begin
  34.   If IsConnected Then
  35.     DisconnectFromServer; // avoid memory leaks
  36.   if (Aport=0) then
  37.     if UseSSL then
  38.       Aport:=443
  39.     else
  40.       Aport:=80;
  41.   G:=GetSocketHandler(UseSSL);
  42.   if FUnixSocketPath = '' then
  43.     FSocket:=TInetSocket.Create(AHost,APort,G)
  44.   else
  45.     WriteLn(FUnixSocketPath);
  46.     FSocket:=TUnixSocket.Create(FUnixSocketPath);
  47.  
  48.   try
  49.     if FIOTimeout<>0 then
  50.       FSocket.IOTimeout:=FIOTimeout;
  51.     if FConnectTimeout<>0 then
  52.       FSocket.ConnectTimeout:=FConnectTimeout;
  53.    //     if TypeOf(FSocket) is TInetSocket then
  54.     //  FSocket.Connect;
  55.   except
  56.     FreeAndNil(FSocket);
  57.     Raise;
  58.   end;
  59. end;
  60.  
  61. end.
  62.  

MarkMLl

  • Hero Member
  • *****
  • Posts: 6676
Re: HTTP request over Unix Domain Socket
« Reply #12 on: January 25, 2022, 12:21:58 pm »
I can't help with the HTTP side of it... I'm not even going to start thinking about it since it's outside my field and I've got a lot of other stuff on my plate.

I suggest stepping into TUnixSocket.Create() and seeing what's going wrong, and also looking at the earlier discussion. It might be something silly like a malformed path or access rights to (the file-like named entity implementing) the path, or it might be that that object isn't fully implemented. In extremis read through the earlier thread I cited and try opening the socket as a simple text file.

Most of my experience with UD sockets has involved using them for datagrams, i.e. UDP-like rather than TCP-like. The problems were almost always on the server side (i.e. access rights etc. which necessitated my getting involved with POSIX Capabilities) but in the case of streams (TCP-like) there are also hazards when a connection has been shut down and not properly restarted.

MarkMLl
MT+86 & Turbo Pascal v1 on CCP/M-86, multitasking with LAN & graphics in 128Kb.
Pet hate: people who boast about the size and sophistication of their computer.
GitHub repositories: https://github.com/MarkMLl?tab=repositories

istoica

  • New Member
  • *
  • Posts: 21
Re: HTTP request over Unix Domain Socket
« Reply #13 on: January 25, 2022, 12:57:25 pm »
I have copy-pasted the entire fphttpclient unit and modified the following

  • Modified private `FSocket : TInetSocket;` to `FSocket : TUnixSocket;`
  • Added  private `FUnixSocketPath: String` to `TFPCustomHTTPClient`
  • Added  public `Property UnixSocketPath : String Read FUnixSocketPath Write FUnixSocketPath;`
  • Modified  public `TFPCustomHTTPClient.ConnectToServer` and commented out `FSocket.Connect;` as there is no Connect for TUnixSocket - one could add dummy Connect to TUnixSocket

Now this beauty works properly, I get expected response

Quote
var
  S : String;
begin

  With TFPCustomHTTPClient.Create(nil) do
    try 
      UnixSocketPath := '/tmp/podman.sock';
      S := Get('http://d/v3.0.0/libpod/info');
      WriteLn(S);
    finally
      Free;
    end;

It does the job, but it is so ... unpleasant to copy-paste the entire unit, it would be so cool if this is built-in

MarkMLl

  • Hero Member
  • *****
  • Posts: 6676
Re: HTTP request over Unix Domain Socket
« Reply #14 on: January 25, 2022, 01:13:55 pm »
In that case you'll have to submit a patch or at least make a feature request providing your suggested implementation via https://gitlab.com/freepascal.org/lazarus/lazarus/-/issues

There was something I found I had to do to get basic sockets working... ah yes,

Code: Pascal  [Select][+][-]
  1.   (* This is a reimplementation of the standard Connect() since it doesn't
  2.     support a unix-domain address and truncates it if told to cast it to an
  3.     inet domain address structure.
  4.  
  5.     Test using e.g.  $ nc -lkU ~/dsocket  noting the -k to prevent the listener
  6.     terminating when the writer (this program) closes the connection.
  7.   *)
  8.   Function Connect(Sock:longint;const addr: TUnixSockAddr;var SockIn,SockOut:text):Boolean;
  9.  
  10.  
  11.     Function DoConnect(Sock:longint;const addr: TUnixSockAddr): Boolean;
  12.  
  13.     var
  14.       res: longint;
  15.     begin
  16.       repeat
  17.         res:=fpconnect(Sock,@Addr,SizeOF(TUnixSockAddr)); (* NOTA BENE *)
  18.       until (res<>-1) or (SocketError <> EsockEINTR);
  19.       DoConnect:= res = 0;
  20.     end;
  21.  
  22.  
  23.   begin
  24.     Connect:=DoConnect(Sock,addr);
  25.     If Connect then
  26.        Sock2Text(Sock,SockIn,SockOut);
  27.   end { Connect } ;
  28.  

noting the different address type, which is of course much the same thing that you tackled in your earlier posting.

MarkMLl
MT+86 & Turbo Pascal v1 on CCP/M-86, multitasking with LAN & graphics in 128Kb.
Pet hate: people who boast about the size and sophistication of their computer.
GitHub repositories: https://github.com/MarkMLl?tab=repositories

 

TinyPortal © 2005-2018