Building a packet from scratch (under Windows)

[role_33]

Ok, just lets make it very simple, you have this packet with the data that is sent to the device, just try to open a connection and send the data manually:

Code: Pascal  [Select][+][-]

1. program Project1;
2.  
3. {$mode objfpc}{$H+}
4.  
5. uses
6.   ssockets;
7.  
8. const
9.   {Put in the data you read out of wireshark }
10.   DataToSend: Array[0..18] of Byte = ($00,$00,$00,$00,$00,$00,$00,$00,$00,$00,$00,$00,$00,$00,$00,$00,$00,$00,$00);
11. var
12.   s: TInetSocket;
13.   buff: array[0..1023] of Byte;
14. begin
15.   s:=TInetSocket.Create('IP of your device', Port of your Device);
16.   try
17.     s.Connect;
18.     s.write(DataToSend,Length(DataToSend)));
19.     s.Read(buff,SizeOf(buff));
20.   finally
21.     s.Free;
22.   end;
23. end.

And look what happens. If it works (in that the connection is at least accepted, what happens afterwards I can't tell), you know the device is kind of using one-shot request response schemes. Then it is rather easy.

And we have traffic! Thanks!
No response, but traffic, which is progress.

When I try to run this, the s.connect fails.
It raises an exception (ESocketError / "Connect to <IP> failed")

From inspecting the packet with Wireshark, the main thing I can see right now is that the packet from the original software has:

Code: [Select]

    [Protocols in frame: eth:ethertype:ip:tcp:data]
    [Coloring Rule Name: TCP]
    [Coloring Rule String: tcp]

as opposed tothe one from your code:

Code: [Select]

    [Protocols in frame: eth:ethertype:ip:tcp]
    [Coloring Rule Name: TCP SYN/FIN]
    [Coloring Rule String: tcp.flags & 0x02 || tcp.flags.fin == 1]

Where can I control the flags passed?

J

[Warfley]

You can't, as was said multiple times in this thread, TCP is a stream based protocol. Syn packets are used to setup the stream.

The problem seems to be that your device can only ever hold one stream at a time, meaning when the other, original software is running and has that stream setup, you cannot connect from a different application. This is why it is failing on the connect.

So you have two options, either disconnect the other software (temporarily) and while it's down establish your own stream, or you must intercept the existing stream.

[MarkMLl]

You can't, as was said multiple times in this thread, TCP is a stream based protocol. Syn packets are used to setup the stream.

I resisted the temptation to point that out before you did :-)

There's been tomes written on the innards of TCP, and the significance of the packet flags (SYN etc.), connection parameters (DF etc.) and RFC-recognised options (window size etc.). However by and large THE APPLICATION PROGRAMMER DOES NOT NEED TO GET INVOLVED WITH THAT STUFF.

The problem seems to be that your device can only ever hold one stream at a time, meaning when the other, original software is running and has that stream setup, you cannot connect from a different application. This is why it is failing on the connect.

Makes sense. IOW, either the device is using a fairly low-end embedded OS with absolutely minimal multitasking capability, or it's running e.g. Linux but the backlog parameter passed to the listen() function is 1. Again, there's nothing the programmer at the client end can do to change this.

MarkMLl

[role_33]

Thank you, gents!

@Warfley: the only software running on the computer attached to the device was my Lazarus test program. The other software had not been run on either my computer nor the device after I powered it up this morning - meaning the only stream was the one I created by using your code.

Now knowing what to look for, I checked the logs again.

- I started the other piece of software and established the connection (literally, press "Connect").
- I can see a SYN package being sent.

Specifically:

Code: [Select]

  [C] -> [Broadcast] ARP packet
  [H] -> [C] ARP packet
  [C] -> [H] ping
  [H] -> [C] ping
  [C] -> [H] ping
  [H] -> [C] ping
  [H] -> [C] ARP packet
  [C] -> [H] ARP packet
  [C] -> [H] SYN packet
  [H] -> [C] SYN, ACK packet
  .
  .
  .

Where  [C]: client/PC, and [H]: host/device

I just tested something, though: just creating the TInetSocket and freeing it immediately after:

Code: Pascal  [Select][+][-]

1. procedure TForm1.Test;
2. var
3.   s: TInetSocket;
4. begin
5.   s:=TInetSocket.Create('192.168.xxx.xxx', <Port>);
6.   s.Free;
7. end;  
8.  

... and this generates more traffic than just establishing the connection on the original software.
Perhaps the device is expecting those pings? I dunno...

Anyway: I'll do some more tests before I bug you guys again.

I appreciate your help, a lot!

Thanks!

J

[MarkMLl]

SYN and SYN/ACK are part of the "three-way handshake" which establishes a TCP connection. There's always the possibility that the supplier-provided client and server were "being clever", but by and large I'd expect you to only need to look at the data messages.

I almost wrote "packets" there, but again must emphasise that TCP is not a packet-oriented protocol.

MarkMLl

[Warfley]

Specifically:

Code: [Select]

  [C] -> [Broadcast] ARP packet
  [H] -> [C] ARP packet
  [C] -> [H] ping
  [H] -> [C] ping
  [C] -> [H] ping
  [H] -> [C] ping
  [H] -> [C] ARP packet
  [C] -> [H] ARP packet
  [C] -> [H] SYN packet
  [H] -> [C] SYN, ACK packet
  .
  .
  .

Where  [C]: client/PC, and [H]: host/device

Just as a quick explaination. ARP is the protocol thats used to figure out which machine has which local IP address. Basically whenever you try to connect to some device with a certain IP address your computer starts with broadcasting onto the network "Hey who'se got the IP for XXX", and the device with that IP then answers. Basically screaming into the void and hoping someone responds. This is exactly what you see before the first ping. Then before setting up the TCP connection, the same happens again, just instead of a broadcast, because the PC already knows the target, they just ask directly: "Hey do you still have that IP" and get a response.
So just consider these as noise.

... and this generates more traffic than just establishing the connection on the original software.
Perhaps the device is expecting those pings? I dunno...

I mean it wouldn't be something I would generally call a good practice (ICMP packets are often by many routers, active switches and so on, depriorized or completely dropped) so relying on ICMP communication is really wonky, that said, you are talking about specialized embedded hardware, and I have seen all kinds of stuff where the engineers have tried to "be smart".
Maybe look into the Ping packages, maybe they do some special stuff in there. ICMP messages can have additional data, which is not part of the protocol itself, but maybe the device does some special stuff there.

Doing ICMP is not that trivial, as it requires to work with IP_RAW sockets, and I don't think the fcl has a native implementation. I think indy supports ICMP sockets, so you may look into that direction.

[MarkMLl]

Maybe look into the Ping packages, maybe they do some special stuff in there. ICMP messages can have additional data, which is not part of the protocol itself, but maybe the device does some special stuff there.

Doing ICMP is not that trivial, as it requires to work with IP_RAW sockets, and I don't think the fcl has a native implementation. I think indy supports ICMP sockets, so you may look into that direction.

Also: Does Wireshark explicitly describe those as pings (i.e. ICMP echos), or are you mistakenly assuming that anything ICMP is a ping? There's several commands other than ping, and some of those are potentially important.

I think I've used a DLL which gave me fairly painless ping access back in my Windows days. It's more of a problem on unix (i.e. including Linux) since a program that wants to send an ICMP message (of any type) generally needs elevated capabilities.

MarkMLl

[role_33]

Hello Gents,

Thank you for the suggestion to look into the PING packets.
Wireshark does identify them as ping echo, request/reply ("eth.ethertype.ip.icmp.data").

I sent a PING command from the command prompt, and from what I can see the packets do look slightly different, but I'm not sure how significant that is:
- the TTL on the ping from my windows machine is 128, but the one from the old software is 255
- the 32 byte data on the ping from CMD is the typical alphabet sequence, but from the old software it's all zeros

Besides these two differences, everything else looks the same (minus the timestamps, sequence numbers, etc.)

Just because it's not Friday yet (i.e. there's still some weekly pain to be inflicted) , shall I assume that customizing the PING command (using e.g. Synapse) is not an option?

Thx!

J

[MarkMLl]

The TTL limits the number of network hops so is unlikely to be relevant. The payload is simply cruft and is also unlikely to be relevant.

Having said that a ping could /potentially/ be used for a wakeup/knocking sequence, but it's not very likely.

MarkMLl

[Warfley]

What you can try is very simply the following: use the windows ping util to send two pings and then try to connect and look what happens:

Code: Pascal  [Select][+][-]

1. var
2.   s:TInetSocket;
3.   outp:string;
4. begin
5.   { Send two pings }
6.   RunCommand('ping', ['-n', '2', DeviceIP], outp);
7.   { Try to connect }
8.   s:=TInetSocket.Create(DeviceIP,DevicePort);
9.   s.connect;
10.   s.Free;
11. end.

[MarkMLl]

What you can try is very simply the following: use the windows ping util to send two pings and then try to connect and look what happens:

Noting of course your earlier comment about the possibility that the server might only be prepared to handle one connection at a time, and noting also that it might require a "rest" of a minute or two between connections.

It would also be worth having Wireshark active while doing this, and looking at the server's response when the connection is attempted: this might either be a TCP negotiation packet or an ICMP packet (one of the other roles of ICMP is error reporting).

MarkMLl

[role_33]

Hello people,

I hope this is not as stoopid as I feel right now, but:

I went through Warfley's code step by step, looking at Wiresharks at each point, and I noticed that after creating the TInetSocket I already had the packets the old program (the one I want to get rid of) was sending at that point in time.

So, I just removed the "Connect" command... and this thing is working.

Code: Pascal  [Select][+][-]

1. var
2.   s: TInetSocket;
3.   buff: array[0..1023] of Byte;
4. begin
5.   s:=TInetSocket.Create('IP of your device', Port of your Device);
6.   try
7.     //s.Connect; // -> Commenting this line out does the trick
8.     s.write(DataToSend,Length(DataToSend)));
9.     s.Read(buff,SizeOf(buff));
10.   finally
11.     s.Free;
12.   end;
13. end.
14.  

I still need to do a bunch of things (one of the commands I need to implement is "send me a lot of data", so I need to implement receiving that data). But IT WORKS.

Thank you, Thank you, Thank you!!!

J

[MarkMLl]

I still need to do a bunch of things (one of the commands I need to implement is "send me a lot of data", so I need to implement receiving that data). But IT WORKS.

The important thing to appreciate is that TCP data is just a stream... a sequence of bytes, not a TStream etc. but without any protocol-defined delimiters.

MarkMLl

[Remy Lebeau]

I feel like this discussion is going around in circles. Either the device/software is using standard TCP, or it is not. Rather than speculating any further, can you post the actual Wireshark capture of a working communication with the device? Then someone can give you a definitive answer as to how to replicate that session with your own code.

[role_33]

Hello Remy,

The data coming out of these devices -and the communication protocol- is considered sensitive by my company, so I'm not free to post it.
If I were to ask, I would bring unwanted unnecessary attention to my testing.

However, it looks like the device is using standard TCP, but with some specific constraints (e.g. no fragmentation).
As someone said somewhere, the main problem is between the chair and the keyboard.

But with Mark and Wharfley's help, things are moving forward.
I now need to capture the packets sent back by the device, to complete a proof-of-concept.

J