[SOLVED] Hole punching - Step by Step - with simple example

[Key-Real]

I managed to setup the first Step:
I have a UDP server which accepts a client connection and now I knew the IP and Port of the clients.

What is the second Step?

[nummer8]

In this thread hole punching is part of the discussion
https://forum.lazarus.freepascal.org/index.php/topic,52110.0.html

[Key-Real]

I'm on the last step:

Code: Pascal  [Select][+][-]

1.  
2. {$MODE OBJFPC}
3. uses
4.   cthreads, Classes, blcksock, sockets, Synautil, SysUtils, nettools;
5.  
6. var
7.  
8.     soket : TUDPBlockSocket;
9.  
10.     clientNumer : string;
11.  
12.     buf : TcharArray;
13.     s : string;
14.  
15.     bytesReceived : longint;
16.  
17.     remoteIP, remotePort : string;
18.  
19.  
20. Type
21.  
22.     TreadThread = class(TThread)
23.     protected
24.  
25.       procedure Execute; override;
26.  
27.     public
28.  
29.       soket : TUDPBlockSocket;
30.       Constructor Create(CreateSuspended : boolean);
31.     end;
32.  
33.     TwriteThread = class(TThread)
34.     protected
35.  
36.       procedure Execute; override;
37.  
38.     public
39.  
40.       soket : TUDPBlockSocket;
41.       Constructor Create(CreateSuspended : boolean);
42.     end;
43.  
44.  
45. constructor TreadThread.Create(CreateSuspended : boolean);
46. begin
47.     inherited Create(CreateSuspended);
48.     FreeOnTerminate := True;
49. end;
50.  
51. procedure TreadThread.Execute;
52. var buf : TcharArray;
53. begin
54.     while (not Terminated) do begin
55.       bytesReceived:= soket.recvBuffer(PChar(@buf), sizeof(TcharArray));
56.       checkError(soket);
57.       writeln(buf);
58.       sleep(100);
59.     end;
60. end;
61.  
62.  
63.  
64.  
65. constructor TwriteThread.Create(CreateSuspended : boolean);
66. begin
67.     inherited Create(CreateSuspended);
68.     FreeOnTerminate := True;
69. end;
70.  
71. procedure TwriteThread.Execute;
72. var buf : TcharArray;
73.     s : String;
74. begin
75.     while (not Terminated) do begin
76.       s:= '->' + clientNumer;
77.       buf:= s;
78.       soket.SendBuffer(PChar(@buf[0]), length(s));
79.       checkError(soket, 'write');
80.       sleep(100);
81.     end;
82. end;
83.  
84.  
85.  
86.  
87.  
88.  
89.  
90. var
91.     readThread : TreadThread;
92.     writeThread : TwriteThread;
93.    
94.  
95. begin
96.  
97.   soket:= TUDPBlockSocket.Create;
98.   soket.bind('0.0.0.0', '9090');
99.   soket.SetRemoteSin(remoteServerIP, '1500');
100.  
101.   checkError(soket);
102.  
103.  
104.   clientNumer:= '1';
105.   if paramcount = 1 then clientNumer:= paramstr(1);
106.  
107.  
108.  
109.   buf[0]:= char(strtoint(clientNumer));
110.   soket.SendBuffer(PChar(@buf[0]), 1);
111.  
112.   sleep(10);
113.  
114.   soket.Free;
115.  
116.  
117.  
118.   soket := TUDPBlockSocket.Create;
119.   soket.CreateSocket;
120.   soket.bind('0.0.0.0', '9090');
121.   checkError(soket);
122.  
123.   while True do begin
124.     bytesReceived:= soket.recvBuffer(PChar(@buf), sizeof(buf));
125.  
126.     if buf[0] = #3 then begin
127.       writeln('Handshake!');
128.       break;
129.     end;
130.  
131.   end;
132.  
133.   soket.CloseSocket;
134.   soket.Free;
135.  
136.  
137.   s:=transcriptBuf2String(buf, 2, bytesReceived - 1);
138.  
139.   s:=copy(s, pos('@', s) + 1, length(s));
140.   remoteIP:=copy(s, 0, pos(':', s) - 1);
141.   remotePort:=copy(s, pos(':', s) + 1, length(s));
142.  
143.   writeln('other clinet is @', remoteIP, ':', remotePort, '!');
144.  
145. //
146. //      At this point I have correct IP and Port of the other Client!!!!!!!
147. //
148. //
149.  
150. //
151. //  now I'm trying to connect and read/write... nothing happens :(     why?
152. //
153.  
154.   Soket:= TUDPBlockSocket.Create;
155.   Soket.CreateSocket;
156.   Soket.bind('0.0.0.0', '9090'); checkError(Soket, 'bind');
157.   Soket.Connect(remoteIP, remotePort); checkError(Soket, 'connect');
158.  
159.   readThread:=TreadThread.Create(false);
160.   readThread.soket:=soket;
161.   readThread.Start;
162.  
163.   writeThread:= TwriteThread.Create(false);
164.   writeThread.soket:=soket;
165.   writeThread.Start;
166.  
167.  
168.   readln;
169.  
170.   Soket.CloseSocket;
171.   Soket.Free;
172. end.
173.  
174.  

I think I'm connecting wrong. How to establish connection?

[Key-Real]

don't think I haven't look at the discussion at

https://forum.lazarus.freepascal.org/index.php/topic,52110.45.html

they tell just do

a socket.connect(remoteIP, remotePort);

this doesn't work

pls help

[Warfley]

The important part about hole punching is that you need a two way handshake. Basically when you punch the initial hole, the router registers it with a target address, so when then a package comes for that port, that is not from that target address, it will be ignored by the router.
So what you need to do is after punching the first hole, is to then punch the same hole for the new target.

This could look like this:

Code: Text  [Select][+][-]

1. [Alice]             [Server]             [Bob]
2.    |                    |                  |
3.    +--register(alice)-->|                  |
4.    |                    |                  |
5.    |                    |<--request(alice)-+
6.    |                    |                  |
7.    |<-----addr(Bob)-----+---addr(alice)--->|
8.    |                                       |
9.    +------------------punch--------------->|
10.    |<-----------------punch----------------+
11.    |                                       |
12.    |<--------------Communicate------------>|

The "punch" here is just an UDP message whose contents does not matter (it may or may not be blocked by the nat, depending if the punch of the other party has been successful first).
Communication is only possible after both punches went through their respective nat.

It's important that for all operations you always use the same private network port. So you always bind your socket to a fixed port, this way it's ensured that you can reuse the same output port.

[Key-Real]

ok, at the first step I created a server on witch I register Alice IP+Port (line 3)

why on Your Diagram in line 5 You write "request (Alice)? Should I not get the IP+Port of Bob like in step1?
I would have bough addresses, so I can do line 7.

How would line 9 looks in code?

[Warfley]

In this scenario Alice is a kind of server waiting for a connection request. Bob then wants to connect to alice.
The important part is that alice needs the address (IP + Port) of Bob and Bob needs the address (IP + Port) from alice.

Then both parties must send at least one punching message, which will probably be dropped by the other parties NAT. After these two messages they can communicate.

This may look like this (pseudo code, because I don't know how synapse works from the top of my head):
Alice:

Code: Pascal  [Select][+][-]

1. socket.bind('0.0.0.0', LocalPort);
2. repeat
3.   socket.sendTo('register: alice', ServerAddr);
4.   response := socket.recvFrom(ServerAddr, Timeout);
5. until not response.DidTimeout; // Until we get the address from the server
6.  
7. BobAddr := response.Contents; // get the address of bob that the server send us
8.  
9. repeat
10.   socket.sendTo('connect: alice', BobAddr); // functions as hole punch method
11.   BobMessage := socket.recvFrom(BobAddr, Timeout);
12. until not BobMessage.DidTimeout;
13.  
14. // We now have received a message from bob, so we know that he can receive messages from us
15.  

Bob:

Code: Pascal  [Select][+][-]

1. socket.bind('0.0.0.0', LocalPort);
2. repeat
3.   socket.sendTo('request: alice from bob', ServerAddr);
4.   response := socket.recvFrom(ServerAddr, Timeout);
5. until not response.DidTimeout; // Until we get the address from the server
6.  
7. AliceAddr := response.Contents; // get the address of bob that the server send us
8.  
9. repeat
10.   socket.sendTo('connect: bob', BobAddr); // functions as hole punch method
11.   AliceMessage := socket.recvFrom(BobAddr, Timeout);
12. until not AliceMessage.DidTimeout;
13.  
14. // We now have received a message from alice, so we know that he can receive messages from us
15.  

Server:

Code: Pascal  [Select][+][-]

1. socket.bind('0.0.0.0', LocalPort);
2. repeat
3.   Message := socket.recvFrom(Any);
4.   if Message.isRegister then
5.     RegisteredClients[Message.ClientID] := Message.Addr
6.   else if Message.IsRequest then
7.   begin
8.     TargetAddr := RegisteredClients[Message.TargetClientID];
9.     socket.sendTo('addr: ' + Message.ClientID + Message.Addr, TargetAddr); // send Bobs addr to alice
10.     socket.sendTo('addr: ' + Message.TargetClientID + TargetAddr, Message.Addr); // send Alices addr to bob
11.   end;
12. until Stopped;
13.  

[ginoo]

HI,
I think by the end of next month I will try to implement it too with the TCP protocol. I've never done such an implementation even though I needed it. If I find difficulties I will contact you at this post.
Thanks experts.

[Warfley]

Because TCP is a connection based protocol, many routers track the connection state and only allow for messages of that connection. This means you can't just holepunch your way through. At least not reliable through most routers

So what you should do is do UDP hole punching and build a connection based protocol on top of it (like QUIC)

[Key-Real]

SERVER:

Code: Pascal  [Select][+][-]

1. {$MODE OBJFPC}{$H+}
2. uses
3.   cthreads, Classes, blcksock, sockets, Synautil, SysUtils, HTTPSend, synaip, Process, synamisc, nettools;
4.  
5. type
6.     TclientInfo = record
7.                     ip:string;
8.                     port:string;
9.                   end;
10. var
11.     soket : TUDPBlockSocket;                             
12.     clientA, clientB : TclientInfo;
13.     s : string;
14.  
15. procedure receive;
16. begin
17.     repeat
18.         if soket.CanRead(3000) then begin
19.             soket.RecvPacket (3000);
20.             checkError(soket);
21.             break;
22.         end;    
23.     until false;
24.     sleep(100);
25. end;
26.  
27. begin
28.     soket:= TUDPBlockSocket.Create;
29.     soket.bind('0.0.0.0', '1500');
30.     checkError(soket);
31.  
32.     writeln('Waiting for A');
33.     receive;
34.     clientA.ip:= soket.GetRemoteSinIP;
35.     clientA.port:= inttostr(soket.GetRemoteSinPort);
36.     writeln('Client A at ', clientA.ip, ':', clientA.port);    
37.  
38.    
39.     writeln('Waiting for B');
40.     receive;
41.     clientB.ip:= soket.GetRemoteSinIP;
42.     clientB.port:= inttostr(soket.GetRemoteSinPort);
43.     writeln('Client B at ', clientB.ip, ':', clientB.port);    
44.  
45.  
46.     writeln('send to clientA @', clientA.ip, ':', clientA.port);
47.     soket.connect(clientA.ip, clientA.port);
48.     s:=clientB.ip + ':' + clientB.port;
49.     soket.SendString(s);
50.    
51.     writeln('send to clientB @', clientB.ip, ':', clientB.port);
52.     soket.connect(clientB.ip, clientB.port);
53.     s:=clientA.ip + ':' + clientA.port;
54.     soket.SendString(s);
55.  
56.  
57.     soket.CloseSocket;
58.     soket.Free;
59. end.
60.  

CLIENT:

Code: Pascal  [Select][+][-]

1. {$MODE OBJFPC}
2. uses
3.   cthreads, Classes, blcksock, sockets, Synautil, SysUtils, nettools;
4.  
5. var
6.     soket : TUDPBlockSocket;
7.  
8.     clientNumber : string;
9.     s : string;
10.  
11.     remoteIP, remotePort : string;
12. begin
13.   soket:= TUDPBlockSocket.Create;
14.   soket.bind('0.0.0.0', '9090');
15.   soket.connect('178.254.3.95', '1500');
16.   checkError(soket);
17.  
18.  
19.   clientNumber:= '1';
20.   if paramcount = 1 then clientNumber:= paramstr(1);        // start with parameter    '2' for Client B
21.  
22.   soket.SendString(clientNumber);
23.  
24.   repeat
25.     if soket.CanRead(3000) then begin
26.       s:= soket.RecvPacket (30000);
27.       break;
28.     end;
29.     sleep(100);
30.   until false;
31.  
32.   remoteIP:= copy(s, 1, pos(':',s) - 1);
33.   remotePort:= copy(s, pos(':',s) + 1, length(s));
34.  
35.   writeln('punching ', remoteIP, ':', remotePort);
36.  
37. //
38. //
39. // it works till here
40. //
41. //
42.  
43.   soket.connect(remoteIP, remotePort);
44.   checkError(soket);
45.   repeat
46.     soket.SendString('connect from ' + clientNumber);
47.     if soket.CanRead(3000) then begin
48.       s:= soket.recvPacket(3000);
49.       writeln(s);
50.       break;
51.     end;
52.   until false;
53.  
54.   writeln('connected');
55.  
56.   Soket.CloseSocket;
57.   Soket.Free;
58. end.
59.  

I translated Your pseudo code to synapse.
I'm getting the addresses of the client the right way.

Can't punch the hole


Can You pls correct the code?

[ginoo]

Hello @Key-Real,

unfortunately now I don't have time to try, later in a few weeks I'll try too. It is not certain that it will succeed, but we hope ...

[Key-Real]

@Warfley:

ok as I understand You wanna this:
(one way punch first)

ALLICE:

Code: Pascal  [Select][+][-]

1. {$MODE OBJFPC}
2. uses
3.     blcksock, sockets, SysUtils, nettools;
4. var
5.     soket : TUDPBlockSocket;
6.     s : string;
7.     remoteIP, remotePort : string;
8. begin
9.  
10.     writeln('Send to Server...');
11.     soket:= TUDPBlockSocket.Create;
12.     soket.bind('0.0.0.0', '9090');
13.     soket.connect('178.254.3.95', '1500');
14.     checkError(soket);
15.     soket.SendString('Allice');
16.     soket.Free;
17.  
18.  
19.  
20.     writeln('Receive from Server...');
21.     soket:= TUDPBlockSocket.Create;
22.     soket.bind('0.0.0.0', '9090');
23.     repeat
24.       if soket.CanRead(3000) then begin
25.         s:= soket.RecvPacket (3000);
26.         break;
27.       end;
28.       sleep(100);
29.     until false;
30.     remoteIP:= copy(s, 1, pos(':', s) - 1);
31.     remotePort:= copy(s, pos(':', s) + 1, length(s));
32.     soket.Free;
33.  
34.  
35.  
36.     writeln('Punching ', remoteIP, ':', remotePort, '...');
37.     soket:= TUDPBlockSocket.Create;
38.     soket.bind('0.0.0.0', '9090');
39.     repeat
40.       soket.connect(remoteIP, remotePort);
41.       checkError(soket);
42.  
43.       soket.SendString('from Allice');
44.       sleep(100);
45.     until false;
46.  
47.     Soket.Free;
48. end.
49.  

BOB:

Code: Pascal  [Select][+][-]

1. {$MODE OBJFPC}
2. uses
3.     blcksock, sockets, sysutils, nettools;
4.  
5. var
6.     soket : TUDPBlockSocket;
7.  
8.     s : string;
9. begin
10.  
11.     writeln('Send to Server...');
12.     soket:= TUDPBlockSocket.Create;
13.     soket.bind('0.0.0.0', '9090');
14.     soket.connect('178.254.3.95', '1500');
15.     checkError(soket);
16.  
17.     soket.SendString('Bob');
18.    
19.     soket.Free;
20.  
21.  
22.  
23.     writeln('Wait from Allice...');
24.     soket:= TUDPBlockSocket.Create;
25.     soket.bind('0.0.0.0', '9090');
26.     repeat
27.       if soket.CanRead(3000) then begin
28.         s:= soket.RecvPacket (3000);
29.         break;
30.       end;
31.       sleep(100);
32.     until false;
33.  
34.     writeln(s);
35.  
36.  
37.   Soket.Free;
38. end.
39.  

SERVER:

Code: Pascal  [Select][+][-]

1. {$MODE OBJFPC}{$H+}
2. uses
3.     blcksock, sockets, Synautil, SysUtils, HTTPSend, synaip, Process, synamisc, nettools, crt;
4.  
5. type
6.     TclientInfo = record
7.                     ip:string;
8.                     port:string;
9.                   end;
10. var
11.     soket : TUDPBlockSocket;                             
12.     clientAllice, clientBob : TclientInfo;
13.     s : string;
14.  
15. begin
16.    
17.     soket:= TUDPBlockSocket.Create;
18.     soket.bind('0.0.0.0', '1500');
19.     checkError(soket);
20.  
21.     writeln('Waiting for Allice...');
22.     repeat s:= soket.RecvPacket(3000); until s = 'Allice';
23.     clientAllice.ip:= soket.GetRemoteSinIP;
24.     clientAllice.port:= inttostr(soket.GetRemoteSinPort);
25.     writeln('Allice at ', clientAllice.ip, ':', clientAllice.port);  
26.  
27.     writeln('Waiting for Bob...');
28.     repeat s:= soket.RecvPacket(3000); until s = 'Bob';
29.     clientBob.ip:= soket.GetRemoteSinIP;
30.     clientBob.port:= inttostr(soket.GetRemoteSinPort);
31.     writeln('Bob at ', clientBob.ip, ':', clientBob.port);  
32.  
33.     soket.Free;
34.  
35.  
36.     soket:= TUDPBlockSocket.Create;
37.     soket.bind('0.0.0.0', '1500');
38.  
39.     writeln('send to Allice ', clientBob.ip, ':', clientBob.port);
40.     soket.connect(clientAllice.ip, clientAllice.port);
41.     s:=clientBob.ip + ':' + clientBob.port;
42.     soket.SendString(s);
43. {
44.     writeln('send to Bob ', clientAllice.ip, ':', clientAllice.port);
45.     soket.connect(clientBob.ip, clientBob.port);
46.     s:=clientAllice.ip + ':' + clientAllice.port;
47.     soket.SendString(s);
48. }
49.     readln;
50.     soket.Free;
51. end.
52.  

YOU WRONG!!!!!!!!

this can't work because in line 25 of BOB i do a bind and it gets no connections from outside!

pls write back

[Warfley]

Try sending more packages from both Alice and Bob until you get through (the timeout of 3 seconds is way to high, make it a few Ms or so in your punching loop).

Also note that both parties, Alice and Bob need to punch a hole if both are behind a nat. And as a last note hole punching does not work with all nats in all configurations, e.g. when your punching message is discarded by the other nat, it may send an ICMP response, which most nats ignore, but some may use this event to close the punched hole. In those cases you just need to try a few times in rapid succession to trying to send messages on both sides that make it through in the small window of time the hole is open (as soon as both have received a message the hole will stay open).

The problem is all routers have different implementations of their nats, and what works with one may not work with another. After all hole punching is a hack, and not really intended behavior.

currently I have no setup to test it myself, but here is an example C code that I know from a few years ago works ok: https://github.com/mwarning/UDP-hole-punching-examples/tree/master

As you can see it just does register itself at the server, and then when the server sends the address of a new peer it just spams a few messages until the connection can be established.

You should also be able to translate this quite easily to pascal, either using the sockets u it (which provides the same sockets API used by the C code, so you can basically copy it's flow 1-1), or use the more abstracted synapse classes and methods

[Key-Real]

This doesn't work
https://github.com/mwarning/UDP-hole-punching-examples/tree/master

I compiled it with GCC on both machines.
It connects to the server and get the addresses but can't receive messages from each other.

I mean Skype can do this / Games do this, why we can't?

[rvk]

I mean Skype can do this / Games do this, why we can't?

Are you sure Skype (and those games) really make a direct connection with each other (in your situation)?
It could be that an intermediate server is used (if hole punching fails, Skype will use a central server).

don't think I haven't look at the discussion at
https://forum.lazarus.freepascal.org/index.php/topic,52110.45.html
they tell just do

a socket.connect(remoteIP, remotePort);

this doesn't work

I was part of that discussion. I'm sure that's not the only thing function which was called.
Did you try those examples? For us it worked.
Although it was not my code and it has some clutter which you wouldn't need.
But did you test it?

This doesn't work
https://github.com/mwarning/UDP-hole-punching-examples/tree/master
I compiled it with GCC on both machines.
It connects to the server and get the addresses but can't receive messages from each other.

It says in the README.md there:

The examples have not been successfully tested and are probably broken!