Recent

Author Topic: Indy Client in a Thread  (Read 5817 times)

JimKueneman

  • Full Member
  • ***
  • Posts: 220
Indy Client in a Thread
« on: October 26, 2023, 04:16:43 pm »
I am running an Indy TCP client in a thread.  I am getting a random shut down of the socket during data transfer.  I am not getting any exceptions when it does it.  The thread shuts down because I am using this logic.. is this correct?  Should I be using Connected for the test that something is wrong or could the connection be shut down and restarted in the background and the next time I tried to send/receive it will be automatically reconnected?

Code: Pascal  [Select][+][-]
  1. while not IsTerminated and idTCPClient.Connected do
  2.    begin
  3.       ....
  4.     end;                                                    
  5.  

Remy Lebeau

  • Hero Member
  • *****
  • Posts: 1322
    • Lebeau Software
Re: Indy Client in a Thread
« Reply #1 on: October 26, 2023, 07:07:56 pm »
I am running an Indy TCP client in a thread.  I am getting a random shut down of the socket during data transfer.  I am not getting any exceptions when it does it.  The thread shuts down because I am using this logic.. is this correct?

Makes sense, if the remote server closes its end of the connection, and your client runs out of data to read from its InputBuffer.  In which case, Connected() would return False, breaking your loop.

Should I be using Connected for the test that something is wrong

Not really, that is not its purpose.  You should just read from the connection and let it block until data arrives or an error is raised.  Just be aware that if the server does close the connection intentionally, you would get an EIdConnClosedGracefully exception raised, and then you would have to handle that, even though it is not a real error.

So, what is the actual intent here?  Do you want your thread to terminate when the connection is closed, or do you want it to keep running and possibly reconnect?

could the connection be shut down and restarted in the background and the next time I tried to send/receive it will be automatically reconnected?

No, you have to explicitly write code to close and reconnect the client.
Remy Lebeau
Lebeau Software - Owner, Developer
Internet Direct (Indy) - Admin, Developer (Support forum)

JimKueneman

  • Full Member
  • ***
  • Posts: 220
Re: Indy Client in a Thread
« Reply #2 on: October 26, 2023, 09:14:14 pm »
Thanks Remy

JimKueneman

  • Full Member
  • ***
  • Posts: 220
Re: Indy Client in a Thread
« Reply #3 on: October 27, 2023, 04:23:20 am »
New strangeness started now that I spent all afternoon trying to understand... this was a all working fine....

TIdTcpServer TidTcpClient

I send a message to the Server from the Client (Text form of a GridConnect message);
The Server receives it and sends it reply (I am using IOHandler.ReadStream/WriteStream if that matters).

I can see these message in WireShark.

The Client on the other hand never sees the message...

I am doing this in the method

Code: Pascal  [Select][+][-]
  1.  
  2. property idTCPClient: TIdTCPClient read FidTCPClient write FidTCPClient;      
  3.  
  4.  
  5. ....
  6.  
  7. procedure TLccEthernetClientThread.OnThreadComponentRun(Sender: TIdThreadComponent);
  8. var
  9.   iData: Integer;
  10.   LocalDataArray: TLccDynamicByteArray;
  11.   GridConnectStrPtr: PGridConnectString;
  12.   MessageStr: String;
  13. begin
  14.      
  15.   if not IdTCPClient.IOHandler.InputBufferIsEmpty then
  16.   begin
  17.     ReceiveStreamConnectionThread.Position := 0;
  18.     ReceiveStreamConnectionThread.Size := 0;    // Would this set Postion too?
  19.     idTCPClient.IOHandler.ReadStream(ReceiveStreamConnectionThread, idTCPClient.IOHandler.InputBuffer.Size);
  20.  
  21.     ..... bunch of useful stuff
  22.   end;
  23.  
  24.     // https://stackoverflow.com/questions/64593756/delphi-rio-indy-tcpserver-high-cpu-usage
  25.     // There is another way to do this but with this simple program this is fine
  26.   IndySleep(THREAD_SLEEP_TIME);  
  27. end;        
  28.  

The method is continuously called so it is still running and the connection has not been dropped at either end as I can repeat the process and send a message from the Client to the Server and the Server respond but

Code: Pascal  [Select][+][-]
  1. if not IdTCPClient.IOHandler.InputBufferIsEmpty then
  2.  

  Always returns false.  I am using the same architecture to watch for incoming messages on the Server side as well.   This was working until today.  I am stumped....

Jim
« Last Edit: October 27, 2023, 04:25:34 am by JimKueneman »

JimKueneman

  • Full Member
  • ***
  • Posts: 220
Re: Indy Client in a Thread
« Reply #4 on: October 27, 2023, 04:11:14 pm »
This seems to have solved it for now...

Code: Pascal  [Select][+][-]
  1.  
  2. procedure TLccEthernetClientThread.OnThreadComponentRun(Sender: TIdThreadComponent);
  3. var
  4.   iData: Integer;
  5.   LocalDataArray: TLccDynamicByteArray;
  6.   GridConnectStrPtr: PGridConnectString;
  7.   MessageStr: String;
  8. begin
  9.      
  10.   IdTCPClient.IOHandler.CheckForDataOnSource(1);      <<< Added
  11.   if not IdTCPClient.IOHandler.InputBufferIsEmpty then
  12.   begin
  13.     ReceiveStreamConnectionThread.Position := 0;
  14.     ReceiveStreamConnectionThread.Size := 0;    // Would this set Postion too?
  15.     idTCPClient.IOHandler.ReadStream(ReceiveStreamConnectionThread, idTCPClient.IOHandler.InputBuffer.Size);
  16.  
  17.     ..... bunch of useful stuff
  18.   end;
  19.  
  20.     // https://stackoverflow.com/questions/64593756/delphi-rio-indy-tcpserver-high-cpu-usage
  21.     // There is another way to do this but with this simple program this is fine
  22.   IndySleep(THREAD_SLEEP_TIME);  
  23. end;  
  24.  

Remy Lebeau

  • Hero Member
  • *****
  • Posts: 1322
    • Lebeau Software
Re: Indy Client in a Thread
« Reply #5 on: October 27, 2023, 07:50:43 pm »
The Client on the other hand never sees the message...
...
Code: Pascal  [Select][+][-]
  1. if not IdTCPClient.IOHandler.InputBufferIsEmpty then
  2.  

Always returns false.

That is because the InputBuffer is filled with data only when a read operation is performed on the underlying socket, which CheckForDataOnSource() does.  Indy doesn't read from the socket until you tell Indy to read something.  Most of the higher-level reading methods, like ReadStream(), handle this internally for you, checking and filling the InputBuffer until the requested data becomes available in full.

This seems to have solved it for now...

Typically, you don't need the InputBufferIsEmpty() check, just call a reading method like ReadStream() unconditionally and let it block the calling thread until the requested data arrives, eg:

Code: Pascal  [Select][+][-]
  1. procedure TLccEthernetClientThread.OnThreadComponentRun(Sender: TIdThreadComponent);
  2. var
  3.   ...
  4. begin    
  5.   ReceiveStreamConnectionThread.Size := 0;
  6.   idTCPClient.IOHandler.ReadStream(ReceiveStreamConnectionThread, DesiredSize);
  7.   ...
  8. end;  
  9.  

The CheckForDataOnSource()/InputBufferIsEmpty() approach is mainly used only when either:

  • the calling thread does not want to be blocked and wants to do other things even when data is not available.
  • the calling thread does not know how much data is being sent to it (as in your example).

In the latter case, unless you are writing a proxy or streaming media, not knowing how much data to read is generally a sign of bad protocol design.
Remy Lebeau
Lebeau Software - Owner, Developer
Internet Direct (Indy) - Admin, Developer (Support forum)

JimKueneman

  • Full Member
  • ***
  • Posts: 220
Re: Indy Client in a Thread
« Reply #6 on: October 27, 2023, 09:56:57 pm »
Thanks for the explanation.  The protocol was really aimed at CAN bus transfer but a "cheap" TCP adaptation of it has started to gain traction by sending strings (which I am trying to stop and use a TCP specific protocol).

Thanks again.
Jim

Remy Lebeau

  • Hero Member
  • *****
  • Posts: 1322
    • Lebeau Software
Re: Indy Client in a Thread
« Reply #7 on: October 29, 2023, 07:38:04 pm »
I'm not familiar with CAN bus, but I would be very surprised if it doesn't specify proper data sizes in its payloads.  You should be parsing the CAN bus messages to determine how to read them correctly, not just blinding reading arbitrary amounts of bytes, as your earlier example was doing.
Remy Lebeau
Lebeau Software - Owner, Developer
Internet Direct (Indy) - Admin, Developer (Support forum)

JimKueneman

  • Full Member
  • ***
  • Posts: 220
Re: Indy Client in a Thread
« Reply #8 on: October 30, 2023, 02:00:36 am »
Unfortunately the way the protocol was implemented in TCP it does not.. the payload could be from 0 to 8 bytes and you don't know until you find a null character.    The TCP protocol I am trying to push for will have the ability to read directly into the message header and read how many bytes are in the payload (and will be way bigger than 8 bytes... the way it is not is terribly slow as the messages need to ACK back and forth carrying only up to 8 bytes (a CAN limitation).  Then I can just read a known header size and then a known payload size but till then I can only do what others have implemented.
« Last Edit: October 30, 2023, 02:04:29 am by JimKueneman »

JimKueneman

  • Full Member
  • ***
  • Posts: 220
Re: Indy Client in a Thread
« Reply #9 on: November 01, 2023, 02:19:59 pm »
Remy are

procedure IdTCPServerExecute(AContext: TIdContext);

connections serialized calling this function?  I am having this strange hard to reproduce bug with a stream I am using for a read in the function.  It seems like the Size is changing in the middle of a for loop when I am iterating through it within this function but I have not been able to catch it when I add a bit of code to test for it.

Jim

Remy Lebeau

  • Hero Member
  • *****
  • Posts: 1322
    • Lebeau Software
Re: Indy Client in a Thread
« Reply #10 on: November 01, 2023, 04:40:52 pm »
Remy are

procedure IdTCPServerExecute(AContext: TIdContext);

connections serialized calling this function?

No.  Each connection runs in its own worker thread, so the OnConnect/Execute/Disconnect events run in parallel when multiple clients are connected to the server.

I am having this strange hard to reproduce bug with a stream I am using for a read in the function.  It seems like the Size is changing in the middle of a for loop when I am iterating through it within this function but I have not been able to catch it when I add a bit of code to test for it.

Are you sharing a stream across clients?  If so, you need to protect the stream from concurrent access across threads.
Remy Lebeau
Lebeau Software - Owner, Developer
Internet Direct (Indy) - Admin, Developer (Support forum)

JimKueneman

  • Full Member
  • ***
  • Posts: 220
Re: Indy Client in a Thread
« Reply #11 on: November 01, 2023, 10:06:26 pm »
Ok then that explains it.. thanks.

Jim

 

TinyPortal © 2005-2018