Recent

Author Topic: Synapse TCP/IP client and server  (Read 34195 times)

piola

  • Full Member
  • ***
  • Posts: 118
  • Lazarus 2.2, 64bit on Windows 8.1 x64
Re: Synapse TCP/IP client and server
« Reply #30 on: April 23, 2022, 09:11:56 pm »
Thank you again. I will try and report back whether it works.

My program uses a simple text-based communication protocol. Individual messages may be some 100 kB, but this case is rather rare. Typically, only a few 100 bytes are exchanged between client and server. In my case, it's more important to keep delays small.

I have already tried Indy and it worked quite well. But I had designed the server component not very well, so I decided to rewrite it. I read a lots of forum posts about how to do it properly. I tried the components that are included in FPC itself, but they didn't fit my needs (although at the moment I can't remember what the problem was). Your example seemed to be the best designed one, so I wanted to experiment with that. Please apologize for asking a question here and then, but I want to learn about using threads and networking.

piola

  • Full Member
  • ***
  • Posts: 118
  • Lazarus 2.2, 64bit on Windows 8.1 x64
Re: Synapse TCP/IP client and server
« Reply #31 on: April 24, 2022, 01:16:52 am »
Okay, I have a first feedback:

In utcpsockets.pas line 1270, I had to change if FTCPBase.FActiveOrConnected and FBlockSocket.CanRead(1000) then into if FBlockSocket.CanRead(1000) and FTCPBase.FActiveOrConnected then to avoid a deadlock on server termination.

Obviously, the call to Thread.FBlockSocket.CloseSocket in line 1736 causes FBlockSocket.CanRead in line 1270 to return with True which results in blocking in line 1283: the threadlist cannot be added to at this point because in parallel TServer.Cleanup has a lock on the list.

FActiveOrConnected should be False in the condition in line 1270 (it has been set to False in line 1753), but obviously, the check had been evaluated before while FActiveOrConnected had been True. The change to False occurs while waiting for FBlockSocket.CanRead. By swapping the order of evaluation, this issue seems to be fixed.

Btw: FActiveOrConnected is obiously written (in line 1753) to and read (in line 1270) in different threads. Why is there no need for a critical section?
« Last Edit: April 24, 2022, 02:01:31 am by piola »

kupferstecher

  • Hero Member
  • *****
  • Posts: 583
Re: Synapse TCP/IP client and server
« Reply #32 on: April 24, 2022, 12:21:38 pm »
2. The implementation of TClientThread.Execute seems to be responsible for both receiving data from the client and sending data from the server to the client. However, when the server has data waiting to be sent to the client (which is done in ProcessTask), these data will not be sent immediately, but have a delay because the receive timeout (in RecvMessage) always has to be awaited. How can I improve this so that responses from the server to the client are sent immediately?

I think you could change the timeout to zero, so only data that is already available will be fetched. I just checked the Synapse code, there is also a sleep(0) inside RecvPacket(), so the timeout won't be actually zero, but in a operating system there isn't such thing as "immediately" anyways, so that should be within range. (Perhaps it could be optimized a bit by just polling the available data.)
As I understand GetMem's code, a ping is always expected within the timeout and otherwise disconnects, so this would have to be decoupled e.g. by using the systick to identify if a ping wasn't received in time, seems something like that is included already.

Is it possible without causing trouble to use a separate TBlockSocket for the sending thread? I'm asking because both the receiving and the sending thread will have to use the same ports. I don't want the clients themselves to require two threads too [...].

You can only can establish one connection to the same ports, isn't!? So for a seperate sending TBlockSocket you would need an other connection. And in my understanding you also couldn't use two different threads directly accessing one TBlockSocket for sending and receiving (not thread safe). So to achieve full-duplex communication in Synapse you couldn't block receiving with timeouts, as said before. But I never tested if the library really supports full-duplex, sending data while a big download is done on the same connection.

kupferstecher

  • Hero Member
  • *****
  • Posts: 583
Re: Synapse TCP/IP client and server
« Reply #33 on: April 24, 2022, 12:22:45 pm »
By the way, in the meantime I stumbled over a article that describes the issue about the assumption of packets in TCP that I stated in a message earlier in this thread:
https://www.sobyte.net/post/2021-12/whys-the-design-tcp-message-frame/

PizzaProgram

  • Jr. Member
  • **
  • Posts: 53
  • ...developing Delphi apps since 25 years.
Re: Synapse TCP/IP client and server
« Reply #34 on: April 24, 2022, 01:16:30 pm »
I just wanted to report that:
 - Ararat Synapse's current latest trunk (R260) can handle OpenSSL3 well with TLS 1.3!

Here is an example how to send-and-receive JSON UTF8 strings with REST:
Code: Pascal  [Select][+][-]
  1. function HttpPostGetJSON(const URL: string; var JSON: UTF8String;const TimeOut: integer = 5000): Boolean;
  2. const
  3.     CRLF : UTF8String = #13#10;
  4. var
  5.     HTTP    : THTTPSend;
  6.     Data    : TStringStream;
  7. begin
  8.     HTTP := THTTPSend.Create;
  9.     Data := TStringStream.Create(JSON, TEncoding.UTF8);
  10.     JSON := '';
  11.     HTTP.Timeout := TimeOut;
  12.  
  13.     try
  14.         HTTP.Headers.Add('Content-Type: application/json; charset=UTF-8') ;
  15.         HTTP.Headers.Add('Accept: application/json') ;
  16.  
  17.         Data.Position := 0;
  18.       //Data.SaveToFile('C:\data.json');
  19.  
  20.         HTTP.Document.CopyFrom( Data, 0);
  21.         Data.Size := 0;
  22.         Result := HTTP.HTTPMethod('POST', URL);
  23.         if Result then begin
  24.             if HTTP.Document.Size > 0 then begin
  25.                 Data.LoadFromStream( HTTP.Document );
  26.                 JSON := Data.DataString + CRLF + 'Result=' + HTTP.ResultString + CRLF+ 'Protocol='+HTTP.Protocol ;
  27.                 // Data.UnicodeDataString
  28.             end;
  29.         end;
  30.     finally
  31.         HTTP.Free;
  32.         Data.Free;
  33.     end;
  34. end;

I think this function should be part of tzutil.pas .
Would be great if Synapse trunc would move to GitHub, so we can easier commit improvements too.

The one question remains:
- How to set unique Cert + Priv key ?
Code: Pascal  [Select][+][-]
  1. var
  2.   cert, priv_key: string;
  3. ...
  4.         HTTP.Sock.SSL.Certificate := cert ;
  5.         HTTP.Sock.SSL.PrivateKey  := priv_key;
  6.  
What encoding should they use? Hexa ? Base64?
x86_64-win64 --Win7 PRO 64bit HUN

piola

  • Full Member
  • ***
  • Posts: 118
  • Lazarus 2.2, 64bit on Windows 8.1 x64
Re: Synapse TCP/IP client and server
« Reply #35 on: April 24, 2022, 09:05:06 pm »
I have seized on the idea of GetMem using separate threads of sending and receiving. In my program I use a very slimmed-down variant of the original utcpsockets.pas, but I wanted to share the result so I have re-applied my changes to utcpsockets.pas.

@GetMem, maybe you might want to have a look on it whether the changes concord with your concepts. I tried to be as minimally invasive as possible.

@kupferstecher: I had thought about setting the timeout to zero, too, but that would be equal to polling, won't it? I want to avoid this situation because it consumes CPU-time for nothing.

About the ports, I'd have expected that it won't work. But my tests have been positive so far: the concept seems to work. I use two separate TBlockSockets configured with the same settings so they won't interfere with each other. And because on is only sending and the other one is only receiving, there should be no conflict on the socket either. However, I haven't tested high loads yet.

balazsszekely

  • Guest
Re: Synapse TCP/IP client and server
« Reply #36 on: April 26, 2022, 06:40:00 am »
Hi All,

Sorry for the late response I was away for a few days.

@piola
Quote
maybe you might want to have a look on it whether the changes concord with your concepts. I tried to be as minimally invasive as possible.
I will definitely check out your changes and don't worry about my concepts, feel free to modify the code in any way you like. The fact that you shared your changes it's a good thing. Thanks for that!

@kupferstecher
Quote
By the way, in the meantime I stumbled over a article that describes the issue about the assumption of packets in TCP that I stated in a message earlier in this thread:
https://www.sobyte.net/post/2021-12/whys-the-design-tcp-message-frame/
Thanks for the link, interesting article indeed.

@PizzaProgram
Not related to this thread, however I will update synapse in OPM soon. Is this a new release or still part of trunk? IIRC the last release was 40.1.

PizzaProgram

  • Jr. Member
  • **
  • Posts: 53
  • ...developing Delphi apps since 25 years.
Re: Synapse TCP/IP client and server
« Reply #37 on: April 26, 2022, 09:57:01 am »
@GetMem
Quote
Not related to this thread, however I will update synapse in OPM soon. Is this a new release or still part of trunk? IIRC the last release was 40.1.

It is the Latest Trunk R260: https://sourceforge.net/p/synalist/code/HEAD/tree/
Sadly He did not change the version number from 40.1 to 50.0 or something, while it has major changes. I've suggested him to do that, but received no answer yet: 
https://sourceforge.net/p/synalist/bugs/62/
x86_64-win64 --Win7 PRO 64bit HUN

kupferstecher

  • Hero Member
  • *****
  • Posts: 583
Re: Synapse TCP/IP client and server
« Reply #38 on: April 26, 2022, 01:09:38 pm »
@kupferstecher: I had thought about setting the timeout to zero, too, but that would be equal to polling, won't it? I want to avoid this situation because it consumes CPU-time for nothing.
Yes, I'd call it polling then. But the application main loop also is polling, so the overhead should be very small. And a few CPU cycles each 10ms is nearly nothing.

Quote
I have seized on the idea of GetMem using separate threads of sending and receiving. In my program I use a very slimmed-down variant of the original utcpsockets.pas, but I wanted to share the result so I have re-applied my changes to utcpsockets.pas.

[...]

About the ports, I'd have expected that it won't work. But my tests have been positive so far: the concept seems to work. I use two separate TBlockSockets configured with the same settings so they won't interfere with each other. And because on is only sending and the other one is only receiving, there should be no conflict on the socket either. However, I haven't tested high loads yet.
After quickly checking your code, my understanding is that you use the same socket for both threads, which you pass when creating the threads:
Code: Pascal  [Select][+][-]
  1.   Socket := FBlockSocket.Accept; [...]
  2.   recvCT := TClientThread.Create(FTCPBase, Socket); [...]
  3.   sendCT := TClientThread.Create(FTCPBase, Socket);

And I don't think Synapses sockets are thread save in that way, but I'm not sure.

Sharfik

  • Newbie
  • Posts: 1
Re: Synapse TCP/IP client and server
« Reply #39 on: September 01, 2022, 12:34:48 am »
Can not understand. Why is the list of users separately made a simple list. And why send all the processing of message events for the server to the main thread?

datilas

  • New Member
  • *
  • Posts: 14
Re: Synapse TCP/IP client and server
« Reply #40 on: January 09, 2023, 04:09:26 pm »
using application GUI works great,
but i am trying to create a daemon service
with the daemon service the service activates normally.
but when I connect with the client returns the error:
"Invalid username or password.(10013)"
Has anyone tried using it as a daemon service?
Can someone help me?

....

in console application the same problem happens
« Last Edit: January 10, 2023, 03:53:11 pm by datilas »

Phoenix

  • Jr. Member
  • **
  • Posts: 87
Re: Synapse TCP/IP client and server
« Reply #41 on: February 05, 2023, 04:20:32 pm »
@datilas
I don't know if it helps to identify the problem but I got the same error when I tried to add both server and client parts to the same application. When I enter the correct login data, however, I get error "Invalid username or password (10013)" in the client form while on the server form, after confirming the connection, it immediately disconnects with error "Connection reset by peer (10054)"

about communication with windows services some time ago I read on this link https://stackoverflow.com/questions/4451216/how-to-communicate-with-a-windows-service but it probably has nothing to do with it and I can't confirm it: "..with Windows Vista, services can no longer interact directly with users, i.e., no user interfaces.."

@GetMem
In any case thanks for this implementation which I found useful  :)

Phoenix

  • Jr. Member
  • **
  • Posts: 87
Re: Synapse TCP/IP client and server
« Reply #42 on: February 06, 2023, 09:35:00 pm »
I tested version 2.0.12 but unlike version 2.0.10, the client freezes, if while I send a file I go to send something else as a text message. The server receives the file but the client stops responding.
For the test I used a file slightly larger than a gb while for the text only one word. Always reproducible.

Windows 10 64bit
Laz 2.2.4 64bit FPC 3.2.2


balazsszekely

  • Guest
Re: Synapse TCP/IP client and server
« Reply #43 on: February 06, 2023, 09:52:00 pm »
I tested version 2.0.12 but unlike version 2.0.10, the client freezes, if while I send a file I go to send something else as a text message. The server receives the file but the client stops responding.
For the test I used a file slightly larger than a gb while for the text only one word. Always reproducible.

Windows 10 64bit
Laz 2.2.4 64bit FPC 3.2.2
Hi Phoenix,

@piola made some improvement, see attachment from reply #35.  Does it help? If not I will take a look.

PS: I plan to rewrite the whole thing, with a slightly different paradigm. Unfortunately I don't have much free time.

Phoenix

  • Jr. Member
  • **
  • Posts: 87
Re: Synapse TCP/IP client and server
« Reply #44 on: February 06, 2023, 10:52:08 pm »
Hi @GetMem I replaced the utcpsockets.pas file in your version 12. I can connect but the send doesn't send anything (not even failure messages). If I close the client, I get a double disconnect message on the server.

Out of curiosity I tried the modified client with your original server but it doesn't change the result except that I rightly get only one disconnect message when I close the client.

I know that client and server are designed to run on different executables but with the version already available, is it possible to run them on the same executable? The idea would be that each exe has a chance to become server when the active server is shut down. But the exe that will have the active server does not allow you to connect the clients unfortunately.

In any case version 10 for what I need is sufficient and working  :).
Thank you

 

TinyPortal © 2005-2018