Lazarus

Programming => Networking and Web Programming => Topic started by: Gustavo 'Gus' Carreno on August 09, 2022, 11:51:09 am

Title: What do you reach for when you need non blocking TCP sockets?
Post by: Gustavo 'Gus' Carreno on August 09, 2022, 11:51:09 am
Hey Y'all!!

Like mentioned in the Subject, I'm wondering what is your package or lib of choice when you need to use non-blocking TCP sockets.

I need to make an example that needs to have a Thread polling a connection and I'm confused by the 15m of Googling that I did on the matter.
Most of the results where about the server side, but my example has to be on the client side.
The impression that I got: use LNet. But it's not a solid thing, just a feeling from 15m of bad Googling.

So now, I ask the experts and absorb the onslaught of information (or silence, that can also be a result of this enquiry :) )

Cheers,
Gus
Title: Re: What do you reach for when you need non blocking TCP sockets?
Post by: marcov on August 09, 2022, 12:17:17 pm
Indy can be used nonblocking if you regularly  check for data on source

Code: Pascal  [Select][+][-]
  1.     fclient.IOHandler.CheckForDataOnSource(5);

and then never read more than inputbuffer.size:

Code: Pascal  [Select][+][-]
  1.                      
  2.                     insiz:=iohandler.InputBuffer.Size;
  3.                      iohandler.ReadStream(strmbuf,insiz,false);
  4.  

While Indy 10 does a lot with streams (due to its .NET heritage) you can workaround it by using a tcustommemorystream derivative that just points to your own buffers, with nearly no overhead.

I myself use this for a set of client-servers that do work in threads, but use events to try to reach (relatively ) low latency comms.
Title: Re: What do you reach for when you need non blocking TCP sockets?
Post by: SymbolicFrank on August 09, 2022, 12:27:51 pm
Non-blocking is easy: sleep(1000). Blocking is harder and requires a mutex or WaitForSingleObject.

But why would you want to waste CPU time by polling?
Title: Re: What do you reach for when you need non blocking TCP sockets?
Post by: MarkMLl on August 09, 2022, 12:36:45 pm
Like mentioned in the Subject, I'm wondering what is your package or lib of choice when you need to use non-blocking TCP sockets.

That depends entirely on what you want on top. If it were a custom protocol (i.e. not HTTP etc., and definitely not HTTPS etc.) I'd use the OS's API which in almost all cases is derived from Berkeley Sockets so is fairly consistent, driven by a thread.

The only thing I'd use LNet for is Telnet for debugging support or mainframe emulation, I've wasted too much time trying to debug it when things go wrong- particularly after one end disconnects unexpectedly. And even there I'll write my own support at some point.

MarkMLl

p.s. Since you've mentioned Googling, please don't put FPC version etc. in your sig. As soon as you change that ALL messages that you've posted will show the new sig, meaning that they're worthless to anybody getting here via Google or the Forum's own search.
Title: Re: What do you reach for when you need non blocking TCP sockets?
Post by: Warfley on August 09, 2022, 12:47:39 pm
For Raw TCP I use the raw sockets api. There is no need to use any big chunky external library for something that already provides a well designed cross plattform API such as berkley sockets.

For non blocking it depends on what you want to do. Just non blocking reads and writes can be archived with select/poll or even epoll (Linux) and WaitMultipleObjects (Windows). If you want the whole lifecycle being non blocking (e.g. also accept, connect, etc.) then make the file descriptor non blocking with socket option O_NONBLOCK.

I've wrote a small wrapper around the sockets api, with basically the same functions just less C-ish (so stuff like string support, exceptions etc.): https://github.com/Warfley/PasSimpleSockets/blob/master/simplesockets.pas
This has a simplified version of select/poll with DataAvailable (as output buffers should not be the problem on modern systems, this only concerns if there is data waiting to be read).
Simple example for non blocking sockets (it's UDP but a TCP client would look pretty similar): https://github.com/Warfley/PasSimpleSockets/blob/master/examples/DataPendingTest.pas
Code: Pascal  [Select][+][-]
  1. program DataPendingTest;
  2.  
  3. {$mode objfpc}{$H+}
  4.  
  5. uses
  6.   SimpleSockets;
  7.  
  8. var
  9.   Sock: TSocket;
  10.   dots, i: Integer;
  11.   Msg: TReceiveFromStringMessage;
  12. begin
  13.   Sock := UDPSocket(stDualStack);
  14.   // Listen to any connection
  15.   Bind(Sock, '::0', 1337);
  16.   dots := 0;
  17.   // DataAvailable will wait until data is available, or stop after 1 second
  18.   while not DataAvailable(Sock, 1000) do
  19.   begin
  20.     // When no data available after 1 second show message
  21.     Write(#13'Waiting for Data');
  22.     for i := 0 to dots do
  23.       Write('.');
  24.     Write('  ');
  25.     dots := (dots + 1) mod 3;
  26.   end;
  27.   Msg := ReceiveStrFrom(Sock);
  28.   WriteLn;
  29.   WriteLn('Data Received from ', Msg.FromAddr.Address, ':', Msg.FromPort, ': ', Msg.Data);
  30.   SendStrTo(Sock, Msg.FromAddr, Msg.FromPort, Msg.Data);
  31.   ReadLn;
  32. end.
This checks if data available, and if there was no data within 1 second it will update the console to show it is doing something.

But you don't need such a wrapper library either, if you look at the source code, you will see that the functions are basically just thin wrappers around the berkley sockets functions you could also call directly:
Code: Pascal  [Select][+][-]
  1. function Receive(const ASocket: TSocket; ABuffer: Pointer; MaxSize: SizeInt;
  2.   AFlags: Integer): SizeInt;
  3. begin
  4.   Result := fprecv(ASocket.FD, ABuffer, MaxSize, AFlags);
  5.   if Result = 0 then
  6.     raise EConnectionClosedException.Create('The connection closed')
  7.   else if Result < 0 then
  8.     raise ESocketError.Create(socketerror, 'recv');
  9. end;
It's easy as that. The only thing that is slightly more complex is the DataAvailable function because it uses select and select is complicated (the easier poll is not provided by the fpcs socket or baseunix unit and therefore I had to revert back to select, but poll is usually easier):
Code: Pascal  [Select][+][-]
  1. function DataAvailable(const SocketArray: TSocketArray; TimeOut: Integer
  2.   ): TSocketArray;
  3. var
  4.   FDSet: TFDSet;
  5.   MaxSock: TSocketFD;
  6.   timeval: TTimeVal;
  7.   Ret: LongInt;
  8.   i, WriteHead: Integer;
  9. begin
  10.   Result := nil;
  11.   MaxSock := 0;
  12.   {$IfDef UNIX}fpFD_ZERO{$else}FD_ZERO{$endif}(FDSet);
  13.   for i:=0 to Length(SocketArray) - 1 do
  14.   begin
  15.     MaxSock := Max(MaxSock, SocketArray[i].FD);
  16.     {$IfDef UNIX}fpFD_SET{$else}FD_SET{$endif}(SocketArray[i].FD, FDSet);
  17.   end;
  18.   timeval.tv_sec := TimeOut div 1000;
  19.   timeval.tv_usec := (TimeOut mod 1000) * 1000;
  20.   Ret := {$IfDef UNIX}fpselect{$else}select{$endif}(MaxSock + 1, @FDSet, nil, nil, @timeval);
  21.   if Ret < 0 then
  22.     raise ESocketError.Create(socketerror, 'select');
  23.  
  24.   SetLength(Result, Ret);
  25.   WriteHead := 0;
  26.   for i:=0 to Length(SocketArray) - 1 do
  27.     if {$IfDef UNIX}fpFD_ISSET{$else}FD_ISSET{$endif}(SocketArray[i].FD, FDSet) {$Ifdef Unix}> 0{$Endif} then
  28.     begin
  29.       Result[WriteHead] := SocketArray[i];
  30.       Inc(WriteHead);
  31.     end;
  32. end;
But this also checks if data is available for any of multiple sockets, for just checking one socket is much easier (basically just select result > 0 check)
Title: Re: What do you reach for when you need non blocking TCP sockets?
Post by: MarkMLl on August 09, 2022, 12:57:53 pm
For Raw TCP I use the raw sockets api. There is no need to use any big chunky external library for something that already provides a well designed cross plattform API such as berkley sockets.

If I could broaden things a bit: particularly on unix, bear in mind that there are other types of sockets for local (same-host) connection: unix-domain, FIFO and so on.

If you don't need a networked (i.e. routable) socket then don't use one, and in particular look for viable alternatives to overloading HTTP... I'm seriously considering investigating RDP https://en.wikipedia.org/wiki/Reliable_Data_Protocol for a couple of things with SCTP being another option.

https://forum.lazarus.freepascal.org/index.php/topic,39141.0.html
https://forum.lazarus.freepascal.org/index.php/topic,57706.msg429320.html#msg429320

MarkMLl
Title: Re: What do you reach for when you need non blocking TCP sockets?
Post by: marcov on August 09, 2022, 12:57:56 pm
It's easy as that. The only thing that is slightly more complex is the DataAvailable function because it uses select and select is complicated (the easier poll is not provided by the fpcs socket or baseunix unit and therefore I had to revert back to select, but poll is usually easier):

https://www.freepascal.org/docs-html/rtl/baseunix/fppoll.html
Title: Re: What do you reach for when you need non blocking TCP sockets?
Post by: Warfley on August 09, 2022, 05:50:56 pm
It's easy as that. The only thing that is slightly more complex is the DataAvailable function because it uses select and select is complicated (the easier poll is not provided by the fpcs socket or baseunix unit and therefore I had to revert back to select, but poll is usually easier):

https://www.freepascal.org/docs-html/rtl/baseunix/fppoll.html
Ah you are right, I confused this, it was the windows version that was missing (WSAPoll in winsock2 unit). That said, it can probably be easiely defined as external function, but in my wrapper lib I was just lazy and used what was there (select)
Title: Re: What do you reach for when you need non blocking TCP sockets?
Post by: Gustavo 'Gus' Carreno on August 10, 2022, 03:06:53 am
Hey Y'all!!

First, I want to thank Marco, Mark and Warfley for the very good advice that you guys have put here.

Second, I want to apologise to SymbolicFrank because it seems that the way I worded my post made him believe that I was talking about threads.
Sorry SymbolicFrank, I was talking about TCP Sockets.

All the information provided, albeit being quite welcome, is a bit overwhelming. Not because it was ill presented or is too much, no, the delivery was spot less.
The problem is that I've read some books and my head is now more full of theory compared to the little practice I've had.

I've relied on Synapse for a long time to whisk away the nitty-gritty of having to deal with the differences in Linux/Windows/macOS in terms of sockets.
Now that I'm presented with a problem that, in my opinion, needs something outside the realms of Indy and/or Synapse I find myself lost.

So let's talk a bit more specifics!

Here is the algorithm for the example I want to produce:


In my opinion, informed by some theory books on sockets, it makes sense that I use a non blocking socket for this algorithm.

What do you guys say?
Is this doable with Indy, like Marco answered on Reply #1?
Do I need to go full low level and use something else not provided by the abstractions that Synapse/Indy/LNet facilitate?

Please advise!!

Cheers,
Gus
Title: Re: What do you reach for when you need non blocking TCP sockets?
Post by: MarkMLl on August 10, 2022, 09:15:08 am
Stop right there. When you say "ping", do you literally mean ICMP echo request?

And before we go much further, are you restricting consideration to TCP or are you interested in UDP?

MarkMLl
Title: Re: What do you reach for when you need non blocking TCP sockets?
Post by: Warfley on August 10, 2022, 10:43:48 am
Here is the algorithm for the example I want to produce:

  • Start the thread
  • Send the greeting
  • Send the first ping, reset 5s timer
  • Infinite loop:
     
    • Is there any data to read? Receive data and display it
    • Has the 5s timer dinged? Send Ping, reset 5s timer

In my opinion, informed by some theory books on sockets, it makes sense that I use a non blocking socket for this algorithm.

What do you guys say?
Is this doable with Indy, like Marco answered on Reply #1?
Do I need to go full low level and use something else not provided by the abstractions that Synapse/Indy/LNet facilitate?

Assuming that your ping is a TCP message ping and not a literal ICMP ping, this can be done with poll/select. Using the SimpleSockets wrapper by me to keep it shorter, but can also be easiely done using raw sockets:
Code: Pascal  [Select][+][-]
  1. program Project1;
  2.  
  3. {$mode objfpc}{$H+}
  4.  
  5. uses
  6.   SysUtils, SimpleSockets;
  7.  
  8. var
  9.   client: TSocket;
  10.   Data: String;
  11.   Timer: Integer;
  12.   startTime: QWord;
  13. begin
  14.   Timer := 5000;
  15.   client := TCPSocket(stIPv4);
  16.   try
  17.     Connect(client, '127.0.0.1', 1337);
  18.     SendStr(client, 'Ping');
  19.     startTime := GetTickCount64;
  20.     while True do
  21.       if not DataAvailable(client, Timer) then
  22.       begin
  23.         SendStr(Client, 'Ping');
  24.         startTime:=GetTickCount64;
  25.         Timer := 5000;
  26.       end
  27.       else
  28.       begin
  29.         Data := ReceiveStr(client, 1024); // 1024 max length data can be shorter
  30.         WriteLn(Data);
  31.         // Compute new wait period for ping to the next 5 second mark
  32.         Timer := 5000 - (GetTickCount64 - startTime);
  33.         if Timer < 0 then Timer := 0;
  34.       end;
  35.   finally
  36.     CloseSocket(client);
  37.   end;
  38. end.
  39.  
This will pretty much on the note send every 5 seconds a ping, and if data is received in between will print it
Title: Re: What do you reach for when you need non blocking TCP sockets?
Post by: SymbolicFrank on August 10, 2022, 10:44:11 am
Hi Gus,

I understood what you were asking. I'll explain in a bit more detail.

Unless you turn on sleep or wait mode, your CPU's keep executing instructions non-stop. And to enable multitasking, the OS gives all the processes slices of that time.

So, how does the OS keeps track of which processes get a slice of that time? When your process starts, the OS calls your function "main()". And when that returns, your process is done.

So, if you want your process to keep running, you need a loop. Most of the time a semi-endless one, that runs until it gets terminated. Around and around. But, what if you are waiting for input? What do you do?

Code: Pascal  [Select][+][-]
  1. program Example1;
  2.  
  3. var
  4.   Sock: TSocket;
  5.  
  6. function DoSomething;
  7. var
  8.   s: string;
  9. begin
  10.   s := Sock.Read;
  11.   //
  12. end;
  13.  
  14. begin  // this is your main(), it is called by the OS
  15.   while True do // this is your "endless" loop.
  16.   begin
  17.     if Sock.DataReady then DoSomething;
  18.   end;
  19. end.

Of course, this turns your PC into an expensive space heater and prevents other processes to do something useful. To prevent that, you can limit the amount of time spent looking if data has been received. You can do that by adding "Sleep(xxx)", where xxx determines in how much of a hurry you are to receive that message.

But that still wastes time. The OS has to suspend another process, push its registers, switch the stack, pop yours, page the memory your process needs, if required, etc. It takes time. So, it would be even better if your process gets suspended until data is available. A blocking operation, because your process gets completely blocked from execution in the mean time. But that is exactly what you want in this case!

On a low level, the socket is a piece of hardware, that generates an interrupt when it needs attention, like when data is received. And when you tell the OS that you want to handle that event, it wakes your process when it happens.

Now is it a bit hard to generate an interrupt on a high level, so next to sleeping, you can either use a timer (but that is just a fancy way of sleeping in this case) or you can use WaitForSingleObject/WaitForMultipleObjects, that also suspend (block) your thread, until one or more other threads are done executing. In that way, you can create a thread, have it listen to the socket until something happens and terminate when it does. It sounds a bit complex (and it is often easier to have that thread process the data), but that way you can chain blocking events together.

And, if you don't want to create threads all the time, you can create mutexes instead: objects only one thread can access at any one time and that have an option to wait (block) until it is released.

Greetings, Frank
Title: Re: What do you reach for when you need non blocking TCP sockets?
Post by: Gustavo 'Gus' Carreno on August 10, 2022, 01:39:28 pm
Hey Mark,

Stop right there. When you say "ping", do you literally mean ICMP echo request?

Yeah, you're absolutely right and I made a mess of it again. It's not ICMP related. I'm sorry for the confusion that I spread!!

My example is based on a custom protocol that relies on New Line(Indy's WriteLn on a socket) terminated strings that I did not invent and I'm kinda stuck with.
The Ping is just a message in this custom protocol to make sure the connection is kept alive.
There's also a Pong message.
You're then gonna argue that the triple handshake of a TCP connection and the methods that have been put in it's planning are enough to make the connection be kept alive, and you're right.
The problem is, I'm having to deal with a programmer that barely knows how to program with packed records, let alone OOP(which he publicly states that he hates) and his knowledge of the ins and outs of TCP/IP is below zero. Most of his network code has been done by asking odd questions here and having Remy give him solutions. So more like a Stack Overflow Copy/Paste kinda mess,

And before we go much further, are you restricting consideration to TCP or are you interested in UDP?

As mentioned before, I'm stuck with a custom protocol based on TCP.
I would, quite happily, welcome any thing you could teach me on making this custom protocol work on UDP. That would be grand!!

Cheers,
Gus
Title: Re: What do you reach for when you need non blocking TCP sockets?
Post by: Gustavo 'Gus' Carreno on August 10, 2022, 01:50:03 pm
Hey Warfley,

...
Assuming that your ping is a TCP message ping and not a literal ICMP ping, this can be done with poll/select. Using the SimpleSockets wrapper by me to keep it shorter, but can also be easiely done using raw sockets:
...

Warfley, my man, this is exactly what I had in mind!!!!!

It only needs a way to detect the exit condition, which in a CLI would be hooking to the CTRL+C signal, and in a thread would be the Terminated property being true.
But this is me nitpicking cuz you really did hit the nail bang strait in the head :)

I also quite like the fact that you can put the time out on the block read for 5 seconds, clever. And if some data comes along before the 5s, the next read will block the remaining time. Quite clever indeed.

Could NOT thank you enough for this!!!

Cheers,
Gus
Title: Re: What do you reach for when you need non blocking TCP sockets?
Post by: Gustavo 'Gus' Carreno on August 10, 2022, 02:02:12 pm
Hey SymbolicFrank,

I understood what you were asking. I'll explain in a bit more detail.

I don't want to seem patronizing, or ungrateful or even arrogant, so please don't read my next words in such a context, please!!  :-[

To be quite frank(no pun intended!!), whenever one touches, even just, the rims of threading that person learns about the sleep procedure. It's in every example, because, as you colourfully put it: It would turn your CPU into a space heater.
So, yes I know about the need to use the magic thing that makes my thread yield control to the CPU scheduler.
So in essence, twice, you did not understood my question.

For that I must apologise profusely. I made a mess of it and then again about the ICMP Ping thing, so please believe me when I say that I'm really sorry I didn't word it in a better way!!!

Nonetheless you made a brilliant effort when you explained the whole concept, so for that I think you deserve some praise!!
Thanks for the effort!!!

Cheers,
Gus
Title: Re: What do you reach for when you need non blocking TCP sockets?
Post by: SymbolicFrank on August 10, 2022, 02:16:52 pm
Hi Gus,

I just wanted to say that both the timer and socket have an interrupt build in that wakes your thread when something happens. And if you use Synapse, it creates the thread that handles the receiving automatically when the connection is accepted, as long as you use their examples.

But I agree that it was a bit patronizing of me  :P

Greetings, Frank
Title: Re: What do you reach for when you need non blocking TCP sockets?
Post by: MarkMLl on August 10, 2022, 02:52:45 pm
Yeah, you're absolutely right and I made a mess of it again. It's not ICMP related. I'm sorry for the confusion that I spread!!

No apology needed. But if it were ICMP you'd (a) need to use a different API and (b) find that on unix ping and tracert are either setuid root or have a special POSIX attribute to allow them to be run by an unprivileged user.

Code: Text  [Select][+][-]
  1. # getcap `which ping`
  2. /usr/bin/ping = cap_net_raw+ep
  3.  

Quote
You're then gonna argue that the triple handshake of a TCP connection and the methods that have been put in it's planning are enough to make the connection be kept alive, and you're right.

I most definitely am not, since (a) the detail of that is outside the scope of Berkeley Sockets and (b) following nmap's lead and using a handshake as a probe can trigger intrusion detectors (one of our customers once raised this with me, and was impressed when I explained that we had various things in place to ensure that we knew promptly if something happened which would prevent their receiving scheduled reports).

Now jumping back a couple of paras:

Quote
My example is based on a custom protocol that relies on New Line(Indy's WriteLn on a socket) terminated strings that I did not invent and I'm kinda stuck with.

Bear in mind that TCP is a stream protocol, and just because a message is sent with an EOL there's no guarantee that it will be received as a single EOL-terminated message. Particularly for short messages it /usually/ will, but you /will/ eventually hit a situation where the single message sent

Code: [Select]
Test message\r\n

is actually read back as two messages

Code: [Select]
Test m
essage\r\n

The result of that is that what you really need to do is have the thread do a blocking read on the open connection, accumulate the input, and only when it has seen an intact message enqueue it and e.g. use Synchronise() to tell the main thread that there's something to be looked at.

Yes, I know: small messages should work. But consider the situation where the connection is broken at some point, an intermediate router accumulates multiple messages, and when the connection comes back up it repacks them with borders different from the original. TCP acks use a sliding window, and instead of OKing specific messages they OK byte ranges: the protocol /expects/ the stream to be repackaged depending on the characteristics of intervening hops.

UDP (and for that matter ICMP) does respect boundaries... but determining the end-to-end MTU (Maximum Transmission Unit) can be a problem and there is no guarantee that it will stay the same during the lifetime of a connection. But UDP doesn't guarantee delivery, and doesn't guarantee sequential delivery. There's the outline of using UDP to check that a DNS server is live at https://forum.lazarus.freepascal.org/index.php/topic,59807.msg447776.html#msg447776 although this doesn't use a background thread.

SCTP is, like the above, a transport layer protocol. However it both preserves message boundaries and is reliable, unfortunately it's also fairly "heavyweight" and you probably won't find NATting ADSL (etc.) routers that support it (I've run it over a tunnel through a Draytek router, but couldn't over the router itself).

RDP is an obsolete transport layer protocol which preserves message boundaries and has various options which allow it to deliver out-of-sequence messages to the client: the example in the RFC is a debugger which might want to capture a target system's memory but doesn't have to get everything sequentially.

MarkMLl
Title: Re: What do you reach for when you need non blocking TCP sockets?
Post by: sketch on August 10, 2022, 03:48:56 pm
Yeah, you're absolutely right and I made a mess of it again. It's not ICMP related. I'm sorry for the confusion that I spread!!

No apology needed. But if it were ICMP you'd (a) need to use a different API and (b) find that on unix ping and tracert are either setuid root or have a special POSIX attribute to allow them to be run by an unprivileged user.

Code: Text  [Select][+][-]
  1. # getcap `which ping`
  2. /usr/bin/ping = cap_net_raw+ep
  3.  

Regarding Capabilities (cap_net_raw), that is a Linux thing and not Unix.  AIX, Solaris, and HP-UX do not implement it, or to the best of my knowledge have not.  But I last used Solaris 10 and Aix 6.x, so maybe they have added Capabilities (Posix 1003.)  Sorry to be nitpicking  ::)

If IBM and Oracle have added that in AIX and Solaris, then I stand corrected -- and embarrassed.  :-[
Title: Re: What do you reach for when you need non blocking TCP sockets?
Post by: MarkMLl on August 10, 2022, 04:00:04 pm
I said it's /either/ setuid root /or/ it uses POSIX capabilities. I didn't speculate on how compliant IBM et al. were with any particular facet of the POSIX specs >:-)

MarkMLl
TinyPortal © 2005-2018