Indy Client in a Thread

[JimKueneman]

I am running an Indy TCP client in a thread.  I am getting a random shut down of the socket during data transfer.  I am not getting any exceptions when it does it.  The thread shuts down because I am using this logic.. is this correct?  Should I be using Connected for the test that something is wrong or could the connection be shut down and restarted in the background and the next time I tried to send/receive it will be automatically reconnected?

Code: Pascal  [Select][+][-]

1. while not IsTerminated and idTCPClient.Connected do
2.    begin
3.       ....
4.     end;                                                    
5.  

[Remy Lebeau]

I am running an Indy TCP client in a thread.  I am getting a random shut down of the socket during data transfer.  I am not getting any exceptions when it does it.  The thread shuts down because I am using this logic.. is this correct?

Makes sense, if the remote server closes its end of the connection, and your client runs out of data to read from its InputBuffer.  In which case, Connected() would return False, breaking your loop.

Should I be using Connected for the test that something is wrong

Not really, that is not its purpose.  You should just read from the connection and let it block until data arrives or an error is raised.  Just be aware that if the server does close the connection intentionally, you would get an EIdConnClosedGracefully exception raised, and then you would have to handle that, even though it is not a real error.

So, what is the actual intent here?  Do you want your thread to terminate when the connection is closed, or do you want it to keep running and possibly reconnect?

could the connection be shut down and restarted in the background and the next time I tried to send/receive it will be automatically reconnected?

No, you have to explicitly write code to close and reconnect the client.

[JimKueneman]

Thanks Remy

[JimKueneman]

New strangeness started now that I spent all afternoon trying to understand... this was a all working fine....

TIdTcpServer TidTcpClient

I send a message to the Server from the Client (Text form of a GridConnect message);
The Server receives it and sends it reply (I am using IOHandler.ReadStream/WriteStream if that matters).

I can see these message in WireShark.

The Client on the other hand never sees the message...

I am doing this in the method

Code: Pascal  [Select][+][-]

1.  
2. property idTCPClient: TIdTCPClient read FidTCPClient write FidTCPClient;      
3.  
4.  
5. ....
6.  
7. procedure TLccEthernetClientThread.OnThreadComponentRun(Sender: TIdThreadComponent);
8. var
9.   iData: Integer;
10.   LocalDataArray: TLccDynamicByteArray;
11.   GridConnectStrPtr: PGridConnectString;
12.   MessageStr: String;
13. begin
14.      
15.   if not IdTCPClient.IOHandler.InputBufferIsEmpty then
16.   begin
17.     ReceiveStreamConnectionThread.Position := 0;
18.     ReceiveStreamConnectionThread.Size := 0;    // Would this set Postion too?
19.     idTCPClient.IOHandler.ReadStream(ReceiveStreamConnectionThread, idTCPClient.IOHandler.InputBuffer.Size);
20.  
21.     ..... bunch of useful stuff
22.   end;
23.  
24.     // https://stackoverflow.com/questions/64593756/delphi-rio-indy-tcpserver-high-cpu-usage
25.     // There is another way to do this but with this simple program this is fine
26.   IndySleep(THREAD_SLEEP_TIME);  
27. end;        
28.  

The method is continuously called so it is still running and the connection has not been dropped at either end as I can repeat the process and send a message from the Client to the Server and the Server respond but

Code: Pascal  [Select][+][-]

1. if not IdTCPClient.IOHandler.InputBufferIsEmpty then
2.  

  Always returns false.  I am using the same architecture to watch for incoming messages on the Server side as well.   This was working until today.  I am stumped....

Jim

[JimKueneman]

This seems to have solved it for now...

Code: Pascal  [Select][+][-]

1.  
2. procedure TLccEthernetClientThread.OnThreadComponentRun(Sender: TIdThreadComponent);
3. var
4.   iData: Integer;
5.   LocalDataArray: TLccDynamicByteArray;
6.   GridConnectStrPtr: PGridConnectString;
7.   MessageStr: String;
8. begin
9.      
10.   IdTCPClient.IOHandler.CheckForDataOnSource(1);      <<< Added
11.   if not IdTCPClient.IOHandler.InputBufferIsEmpty then
12.   begin
13.     ReceiveStreamConnectionThread.Position := 0;
14.     ReceiveStreamConnectionThread.Size := 0;    // Would this set Postion too?
15.     idTCPClient.IOHandler.ReadStream(ReceiveStreamConnectionThread, idTCPClient.IOHandler.InputBuffer.Size);
16.  
17.     ..... bunch of useful stuff
18.   end;
19.  
20.     // https://stackoverflow.com/questions/64593756/delphi-rio-indy-tcpserver-high-cpu-usage
21.     // There is another way to do this but with this simple program this is fine
22.   IndySleep(THREAD_SLEEP_TIME);  
23. end;  
24.  

[Remy Lebeau]

The Client on the other hand never sees the message...
...

Code: Pascal  [Select][+][-]

1. if not IdTCPClient.IOHandler.InputBufferIsEmpty then
2.  

Always returns false.

That is because the InputBuffer is filled with data only when a read operation is performed on the underlying socket, which CheckForDataOnSource() does.  Indy doesn't read from the socket until you tell Indy to read something.  Most of the higher-level reading methods, like ReadStream(), handle this internally for you, checking and filling the InputBuffer until the requested data becomes available in full.

This seems to have solved it for now...

Typically, you don't need the InputBufferIsEmpty() check, just call a reading method like ReadStream() unconditionally and let it block the calling thread until the requested data arrives, eg:

Code: Pascal  [Select][+][-]

1. procedure TLccEthernetClientThread.OnThreadComponentRun(Sender: TIdThreadComponent);
2. var
3.   ...
4. begin    
5.   ReceiveStreamConnectionThread.Size := 0;
6.   idTCPClient.IOHandler.ReadStream(ReceiveStreamConnectionThread, DesiredSize);
7.   ...
8. end;  
9.  

The CheckForDataOnSource()/InputBufferIsEmpty() approach is mainly used only when either:

• the calling thread does not want to be blocked and wants to do other things even when data is not available.
• the calling thread does not know how much data is being sent to it (as in your example).

In the latter case, unless you are writing a proxy or streaming media, not knowing how much data to read is generally a sign of bad protocol design.

[JimKueneman]

Thanks for the explanation.  The protocol was really aimed at CAN bus transfer but a "cheap" TCP adaptation of it has started to gain traction by sending strings (which I am trying to stop and use a TCP specific protocol).

Thanks again.
Jim

[Remy Lebeau]

I'm not familiar with CAN bus, but I would be very surprised if it doesn't specify proper data sizes in its payloads.  You should be parsing the CAN bus messages to determine how to read them correctly, not just blinding reading arbitrary amounts of bytes, as your earlier example was doing.

[JimKueneman]

Unfortunately the way the protocol was implemented in TCP it does not.. the payload could be from 0 to 8 bytes and you don't know until you find a null character.    The TCP protocol I am trying to push for will have the ability to read directly into the message header and read how many bytes are in the payload (and will be way bigger than 8 bytes... the way it is not is terribly slow as the messages need to ACK back and forth carrying only up to 8 bytes (a CAN limitation).  Then I can just read a known header size and then a known payload size but till then I can only do what others have implemented.

[JimKueneman]

Remy are

procedure IdTCPServerExecute(AContext: TIdContext);

connections serialized calling this function?  I am having this strange hard to reproduce bug with a stream I am using for a read in the function.  It seems like the Size is changing in the middle of a for loop when I am iterating through it within this function but I have not been able to catch it when I add a bit of code to test for it.

Jim

[Remy Lebeau]

Remy are

procedure IdTCPServerExecute(AContext: TIdContext);

connections serialized calling this function?

No.  Each connection runs in its own worker thread, so the OnConnect/Execute/Disconnect events run in parallel when multiple clients are connected to the server.

I am having this strange hard to reproduce bug with a stream I am using for a read in the function.  It seems like the Size is changing in the middle of a for loop when I am iterating through it within this function but I have not been able to catch it when I add a bit of code to test for it.

Are you sharing a stream across clients?  If so, you need to protect the stream from concurrent access across threads.

[JimKueneman]

Ok then that explains it.. thanks.

Jim