Recent

Author Topic: Synapse TCP/IP client and server  (Read 34184 times)

balazsszekely

  • Guest
Re: Synapse TCP/IP client and server
« Reply #15 on: May 20, 2021, 01:45:58 pm »
@kupferstecher
Quote
there's no doubt that Synapse is working very well. The question was if it can be relied on that a buffer/packet sent via TCP ist received as such packet using Synapse's RecvPacket. In my understanding TCP doesn't guarantee that, but only a stream of bytes in the correct order. Just like a Serial port.
Then perhaps you should switch from RecvPacket to another RecvXY(RecvByte, RecvInteger, etc...). In the end all of them rely on RecvBuffer and the low level Recv function.

kupferstecher

  • Hero Member
  • *****
  • Posts: 583
Re: Synapse TCP/IP client and server
« Reply #16 on: May 20, 2021, 05:46:01 pm »
Yes, or using RecvPacket and working with an extra buffer.

balazsszekely

  • Guest
Re: Synapse TCP/IP client and server
« Reply #17 on: May 25, 2021, 12:11:48 pm »
@kupferstecher

I fixed the issue with the AddTask method, now everything should be thread safe(see attached projects). If you find a way to improve RecvPacket please let me know, so I can integrate it in the test application. Thank you!

Gustavo 'Gus' Carreno

  • Hero Member
  • *****
  • Posts: 1112
  • Professional amateur ;-P
Re: Synapse TCP/IP client and server
« Reply #18 on: May 26, 2021, 12:55:04 am »
Hey GetMem,

Do you have this project in a VCS of ANY sort?

I'm not complaining that you give us the source like you're doing now, it's just that you don't give us the source with a different filename, like v1 and v2, well, you get my meaning.

I'm really interested in this code since I'm planing a server of my own and I'm loving that you are reading comments and improving the code, I do, but could we benefit with an historical record?

Many thanks for the code, in whatever form you give to us, though :)

Cheers,
Gus
Lazarus 3.99(main) FPC 3.3.1(main) Ubuntu 23.10 64b Dark Theme
Lazarus 3.0.0(stable) FPC 3.2.2(stable) Ubuntu 23.10 64b Dark Theme
http://github.com/gcarreno

irawan

  • Newbie
  • Posts: 6
Re: Synapse TCP/IP client and server
« Reply #19 on: May 26, 2021, 07:58:08 am »
Hello avra,

there's no doubt that Synapse is working very well. The question was if it can be relied on that a buffer/packet sent via TCP ist received as such packet using Synapse's RecvPacket. In my understanding TCP doesn't guarantee that, but only a stream of bytes in the correct order. Just like a Serial port.

Would be interesting to know, how you dealt with this issue, parsing the data stream or relying on packets?

Regards~
as far as i know, UDP that is not guarantee that every packet is sent and received correctly. TCP/IP DOES guarantee that every packet is sent correctly altough packet sequence is not guaranteed. coz of this feature, many router will cut your packet if you exceded your quota. in case your packet is dropped by the router, your server will send your dropped packets again and again till download completed. on other side, UDP can not be limited, it's only option is dropped only.

PascalDragon

  • Hero Member
  • *****
  • Posts: 5446
  • Compiler Developer
Re: Synapse TCP/IP client and server
« Reply #20 on: May 26, 2021, 08:57:46 am »
as far as i know, UDP that is not guarantee that every packet is sent and received correctly.

Just to clarify: UDP does not guarantee delivery, however if it is delivered, it is delivered correctly (meaning without errors).

TCP/IP DOES guarantee that every packet is sent correctly altough packet sequence is not guaranteed.

Packet sequence is guaranteed with TCP (at the recipient, not during transit!), otherwise communication that relies on the order of the data (e.g. file downloads) wouldn't be possible without additional protocols.

kupferstecher

  • Hero Member
  • *****
  • Posts: 583
Re: Synapse TCP/IP client and server
« Reply #21 on: May 30, 2021, 07:08:36 pm »
If you find a way to improve RecvPacket please let me know, so I can integrate it in the test application.

Hello Getmem,

I checked the Synapse example code for a HTTPServer. There the header is received as string lines, each line is terminated with a line break (CR+LF). In Synapse there is a RecvString Function that calls RecvPacket repeatedly until the line break characters are found. The rest of a packet that is behind the line break is buffered within Synapse and returned the next time RecvBuffer is called. The size of the http-body is part of the headers and the body is then received with RecvBufferEx with predetermined size. This Synapse function calls RecvPacket in a loop until the defined number of bytes is received. The rest of the 'packet' (if the size doesn't exactly fit), again, is buffered back in Synapse. So both http-parts don't rely on any packets but determine the data size to be received on protocol level. And I think such mechanism is required when using TCP.

Packet sequence is guaranteed with TCP (at the recipient, not during transit!), otherwise communication that relies on the order of the data (e.g. file downloads) wouldn't be possible without additional protocols.
As said before, my understanding of TCP is that there are no packets at all exposed to the application but only a byte stream. So your mentioned file download only works if its size is known to the receiver, otherwise it couldn't determine when the transfer is complete. Is this correct?

PascalDragon

  • Hero Member
  • *****
  • Posts: 5446
  • Compiler Developer
Re: Synapse TCP/IP client and server
« Reply #22 on: May 31, 2021, 01:09:33 pm »
Packet sequence is guaranteed with TCP (at the recipient, not during transit!), otherwise communication that relies on the order of the data (e.g. file downloads) wouldn't be possible without additional protocols.
As said before, my understanding of TCP is that there are no packets at all exposed to the application but only a byte stream. So your mentioned file download only works if its size is known to the receiver, otherwise it couldn't determine when the transfer is complete. Is this correct?

On the network level it is composed of packets. The concept of data streams is built upon TCP, but in principle could also be built upon UDP.

And transmitting the length is not the only way to communicate the end of the stream: the sending side can simply terminate the TCP session once all data is send (this is in fact what's happening with HTTP downloads when the server does not transmit the size, though that's rather seldom nowadays).

delphi_coder

  • Newbie
  • Posts: 1
Re: Synapse TCP/IP client and server
« Reply #23 on: September 23, 2021, 01:42:19 am »
As said before, my understanding of TCP is that there are no packets at all exposed to the application but only a byte stream. So your mentioned file download only works if its size is known to the receiver, otherwise it couldn't determine when the transfer is complete. Is this correct?
Here the word packet maybe a bit confusing, packet from your view differs by its meaning in network topic. All data traveling within TCP packets. There is no guarantee that you receive for example 3kb sent packet from the client at once, I think in most cases data checking and using Stream classes or implementing proper buffering is a must.

Ericktux

  • Sr. Member
  • ****
  • Posts: 345
Re: Synapse TCP/IP client and server
« Reply #24 on: November 17, 2021, 05:13:47 am »
excellent project friend getmem, thanks for sharing, two questions:
1. what does "broadcast" and "broadcasting" mean?
2. How can I get clients to talk to other clients and not to the server.

greetings friends  :)

balazsszekely

  • Guest
Re: Synapse TCP/IP client and server
« Reply #25 on: November 17, 2021, 06:08:37 am »
Hi Ericktux,

Quote
1. what does "broadcast" and "broadcasting" mean?
Select one ore more client + right click and you will see :)

Quote
2. How can I get clients to talk to other clients and not to the server.
You can't. This is not implemented. Shouldn't be to difficult to implement though.

piola

  • Full Member
  • ***
  • Posts: 118
  • Lazarus 2.2, 64bit on Windows 8.1 x64
Re: Synapse TCP/IP client and server
« Reply #26 on: April 23, 2022, 01:38:42 pm »
Hello,

I'm just trying your server example and have two points which are not clear to me:

  • TTCPServer.DisconnectClient only removes the client from the list, but it doesn't actually close the client's connection. Is this intended or is there something missing?
  • The implementation of TClientThread.Execute seems to be responsible for both receiving data from the client and sending data from the server to the client. However, when the server has data waiting to be sent to the client (which is done in ProcessTask), these data will not be sent immediately, but have a delay because the receive timeout (in RecvMessage) always has to be awaited. How can I improve this so that responses from the server to the client are sent immediately?

Thanks for your help.

balazsszekely

  • Guest
Re: Synapse TCP/IP client and server
« Reply #27 on: April 23, 2022, 02:39:03 pm »
Hi piola,

Quote
1. TTCPServer.DisconnectClient only removes the client from the list, but it doesn't actually close the client's connection. Is this intended or is there something missing?
Inside DisconnectClient, method Remove is called, where the socket is closed:
Code: Pascal  [Select][+][-]
  1. TClientThread(AThread).FBlockSocket.CloseSocket;

Quote
The implementation of TClientThread.Execute seems to be responsible for both receiving data from the client and sending data from the server to the client. However, when the server has data waiting to be sent to the client (which is done in ProcessTask), these data will not be sent immediately, but have a delay because the receive timeout (in RecvMessage) always has to be awaited. How can I improve this so that responses from the server to the client are sent immediately?
Yes, I agree, this can be a problem. You can solve it by:
1. Creating more worker threads for time consuming operations like file transfer and streaming. Now there are 3 types:
Quote
TBaseType = (btClient, btServer, btServer_AcceptedClient)
You add more like btFileTransfer, btStreaming etc...For streaming you can switch to udp.

2. Create two TClientThread for each connection, one will be responsible for sending, one for receiving data
Both solution has pros and cons, I chose the first one in my application(unfortunately I cannot share the source code).

Quote
Thanks for your help.
Thanks for your comments.

piola

  • Full Member
  • ***
  • Posts: 118
  • Lazarus 2.2, 64bit on Windows 8.1 x64
Re: Synapse TCP/IP client and server
« Reply #28 on: April 23, 2022, 03:07:56 pm »
Hi, thanks for for quick response.

ad 1: I see, this has been a classical problem in my mind  ::) I interpreted the remove as belonging to the list, not to the TTCPServer class - shame on me.

ad 2: I think I will try the second suggestion. This should also simplify the TClientThread's execute method and I can use an event for resuming the "TSendClientThread"'s when data is available for sending.

Is it possible without causing trouble to use a separate TBlockSocket for the sending thread? I'm asking because both the receiving and the sending thread will have to use the same ports. I don't want the clients themselves to require two threads too because this would cause incompatibilities with other tools (in my case, especially telnet-Clients must be able to connect for debugging purposes).

It might also be possible that #2 is just a misuse of your component on my side. I'm using the TTCPServer's instance's SendMessage method for responding. I'm not sure whether this is correct. In my app, I have to separate the server logic from the protocol logic. All examples I have found for servers using TBlockSocket (e.g. a chat server somewhere else in this forum) do the response directly inside the client's Execute method. Of course, this is done for simplicity of the example, and indeed this makes creating a TCP server very simple, but I feel that things quickly get rather complicate when decoupling response handling from the server and taking into account that it might take a moment to prepare the response while everything should keep responsible in the meantime and no packets must get lost.

Overall, your server is very well designed and I hope it will work threadsafe even for a greater number of clients. I always have a some trouble in getting multithreading right, so your example has been of great value for me.
« Last Edit: April 23, 2022, 03:26:03 pm by piola »

balazsszekely

  • Guest
Re: Synapse TCP/IP client and server
« Reply #29 on: April 23, 2022, 04:42:45 pm »
@piola

Quote
It might also be possible that #2 is just a misuse of your component on my side. I'm using the TTCPServer's instance's SendMessage method for responding. I'm not sure whether this is correct. In my app, I have to separate the server logic from the protocol logic. All examples I have found for servers using TBlockSocket (e.g. a chat server somewhere else in this forum) do the response directly inside the client's Execute method. Of course, this is done for simplicity of the example, and indeed this makes creating a TCP server very simple, but I feel that things quickly get rather complicate when decoupling response handling from the server and taking into account that it might take a moment to prepare the response while everything should keep responsible in the meantime and no packets must get lost.
The demo project was meant more for beginners. I designed SendMessage, SendStream and SendFile as a high level functions, so the users don't have to worry about implementation details. The server logic it is separated from the protocol logic via the TTaskList.  You can read more about it here: https://forum.lazarus.freepascal.org/index.php/topic,48677.msg406330.html#msg406330

Quote
so it possible without causing trouble to use a separate TBlockSocket for the sending thread? I'm asking because both the receiving and the sending thread will have to use the same ports. I don't want the clients themselves to require two threads too because this would cause incompatibilities with other tools (in my case, especially telnet-Clients must be able to connect for debugging purposes).
Slowdowns may occur if both the client and server sends large files for example at the same time. As I mentioned in my previous post, i solved this issue by creating more worker threads. However this does require more threads on the client side too. I don't see how you can solve it without extra threads no matter what solution do you choose. Since I don't know any details about you project, unfortunately I cannot give you any useful information.

Quote
Overall, your server is very well designed and I hope it will work threadsafe even for a greater number of clients. I always have a some trouble in getting multithreading right, so your example has been of great value for me.
Each client connection has a separate thread assigned at server side, so you shouldn't face problems even with a larger number of clients. At least in theory...again without knowing the details I cannot answer. I have a similar, not quite the same client/server application which is running for years, but I had to admit the workload on those application is relatively small.


PS: You should also give Indy a try. Many cool stuff are already implemented in Indy.


 

TinyPortal © 2005-2018