Recent

Author Topic: [SOLVED] Hole punching - Step by Step - with simple example  (Read 6018 times)

Key-Real

  • Sr. Member
  • ****
  • Posts: 372
I managed to setup the first Step:
I have a UDP server which accepts a client connection and now I knew the IP and Port of the clients.

What is the second Step?
« Last Edit: June 01, 2023, 06:13:04 pm by Key-Real »

nummer8

  • Full Member
  • ***
  • Posts: 119
Re: Hole punching - Step by Step - with Synapse
« Reply #1 on: May 28, 2023, 12:04:20 pm »
In this thread hole punching is part of the discussion
https://forum.lazarus.freepascal.org/index.php/topic,52110.0.html

Key-Real

  • Sr. Member
  • ****
  • Posts: 372
Re: Hole punching - Step by Step - with Synapse
« Reply #2 on: May 29, 2023, 08:54:19 pm »
I'm on the last step:




Code: Pascal  [Select][+][-]
  1.  
  2. {$MODE OBJFPC}
  3. uses
  4.   cthreads, Classes, blcksock, sockets, Synautil, SysUtils, nettools;
  5.  
  6. var
  7.  
  8.     soket : TUDPBlockSocket;
  9.  
  10.     clientNumer : string;
  11.  
  12.     buf : TcharArray;
  13.     s : string;
  14.  
  15.     bytesReceived : longint;
  16.  
  17.     remoteIP, remotePort : string;
  18.  
  19.  
  20. Type
  21.  
  22.     TreadThread = class(TThread)
  23.     protected
  24.  
  25.       procedure Execute; override;
  26.  
  27.     public
  28.  
  29.       soket : TUDPBlockSocket;
  30.       Constructor Create(CreateSuspended : boolean);
  31.     end;
  32.  
  33.     TwriteThread = class(TThread)
  34.     protected
  35.  
  36.       procedure Execute; override;
  37.  
  38.     public
  39.  
  40.       soket : TUDPBlockSocket;
  41.       Constructor Create(CreateSuspended : boolean);
  42.     end;
  43.  
  44.  
  45. constructor TreadThread.Create(CreateSuspended : boolean);
  46. begin
  47.     inherited Create(CreateSuspended);
  48.     FreeOnTerminate := True;
  49. end;
  50.  
  51. procedure TreadThread.Execute;
  52. var buf : TcharArray;
  53. begin
  54.     while (not Terminated) do begin
  55.       bytesReceived:= soket.recvBuffer(PChar(@buf), sizeof(TcharArray));
  56.       checkError(soket);
  57.       writeln(buf);
  58.       sleep(100);
  59.     end;
  60. end;
  61.  
  62.  
  63.  
  64.  
  65. constructor TwriteThread.Create(CreateSuspended : boolean);
  66. begin
  67.     inherited Create(CreateSuspended);
  68.     FreeOnTerminate := True;
  69. end;
  70.  
  71. procedure TwriteThread.Execute;
  72. var buf : TcharArray;
  73.     s : String;
  74. begin
  75.     while (not Terminated) do begin
  76.       s:= '->' + clientNumer;
  77.       buf:= s;
  78.       soket.SendBuffer(PChar(@buf[0]), length(s));
  79.       checkError(soket, 'write');
  80.       sleep(100);
  81.     end;
  82. end;
  83.  
  84.  
  85.  
  86.  
  87.  
  88.  
  89.  
  90. var
  91.     readThread : TreadThread;
  92.     writeThread : TwriteThread;
  93.    
  94.  
  95. begin
  96.  
  97.   soket:= TUDPBlockSocket.Create;
  98.   soket.bind('0.0.0.0', '9090');
  99.   soket.SetRemoteSin(remoteServerIP, '1500');
  100.  
  101.   checkError(soket);
  102.  
  103.  
  104.   clientNumer:= '1';
  105.   if paramcount = 1 then clientNumer:= paramstr(1);
  106.  
  107.  
  108.  
  109.   buf[0]:= char(strtoint(clientNumer));
  110.   soket.SendBuffer(PChar(@buf[0]), 1);
  111.  
  112.   sleep(10);
  113.  
  114.   soket.Free;
  115.  
  116.  
  117.  
  118.   soket := TUDPBlockSocket.Create;
  119.   soket.CreateSocket;
  120.   soket.bind('0.0.0.0', '9090');
  121.   checkError(soket);
  122.  
  123.   while True do begin
  124.     bytesReceived:= soket.recvBuffer(PChar(@buf), sizeof(buf));
  125.  
  126.     if buf[0] = #3 then begin
  127.       writeln('Handshake!');
  128.       break;
  129.     end;
  130.  
  131.   end;
  132.  
  133.   soket.CloseSocket;
  134.   soket.Free;
  135.  
  136.  
  137.   s:=transcriptBuf2String(buf, 2, bytesReceived - 1);
  138.  
  139.   s:=copy(s, pos('@', s) + 1, length(s));
  140.   remoteIP:=copy(s, 0, pos(':', s) - 1);
  141.   remotePort:=copy(s, pos(':', s) + 1, length(s));
  142.  
  143.   writeln('other clinet is @', remoteIP, ':', remotePort, '!');
  144.  
  145. //
  146. //      At this point I have correct IP and Port of the other Client!!!!!!!
  147. //
  148. //
  149.  
  150. //
  151. //  now I'm trying to connect and read/write... nothing happens :(     why?
  152. //
  153.  
  154.   Soket:= TUDPBlockSocket.Create;
  155.   Soket.CreateSocket;
  156.   Soket.bind('0.0.0.0', '9090'); checkError(Soket, 'bind');
  157.   Soket.Connect(remoteIP, remotePort); checkError(Soket, 'connect');
  158.  
  159.   readThread:=TreadThread.Create(false);
  160.   readThread.soket:=soket;
  161.   readThread.Start;
  162.  
  163.   writeThread:= TwriteThread.Create(false);
  164.   writeThread.soket:=soket;
  165.   writeThread.Start;
  166.  
  167.  
  168.   readln;
  169.  
  170.   Soket.CloseSocket;
  171.   Soket.Free;
  172. end.
  173.  
  174.  



I think I'm connecting wrong. How to establish connection?
« Last Edit: May 30, 2023, 10:30:04 am by Key-Real »

Key-Real

  • Sr. Member
  • ****
  • Posts: 372
Re: Hole punching - Step by Step - with Synapse
« Reply #3 on: May 30, 2023, 10:24:27 am »
don't think I haven't look at the discussion at

https://forum.lazarus.freepascal.org/index.php/topic,52110.45.html

they tell just do

a socket.connect(remoteIP, remotePort);

this doesn't work :(

pls help

Warfley

  • Hero Member
  • *****
  • Posts: 1745
Re: Hole punching - Step by Step - with Synapse
« Reply #4 on: May 30, 2023, 01:57:21 pm »
The important part about hole punching is that you need a two way handshake. Basically when you punch the initial hole, the router registers it with a target address, so when then a package comes for that port, that is not from that target address, it will be ignored by the router.
So what you need to do is after punching the first hole, is to then punch the same hole for the new target.

This could look like this:
Code: Text  [Select][+][-]
  1. [Alice]             [Server]             [Bob]
  2.    |                    |                  |
  3.    +--register(alice)-->|                  |
  4.    |                    |                  |
  5.    |                    |<--request(alice)-+
  6.    |                    |                  |
  7.    |<-----addr(Bob)-----+---addr(alice)--->|
  8.    |                                       |
  9.    +------------------punch--------------->|
  10.    |<-----------------punch----------------+
  11.    |                                       |
  12.    |<--------------Communicate------------>|

The "punch" here is just an UDP message whose contents does not matter (it may or may not be blocked by the nat, depending if the punch of the other party has been successful first).
Communication is only possible after both punches went through their respective nat.

It's important that for all operations you always use the same private network port. So you always bind your socket to a fixed port, this way it's ensured that you can reuse the same output port.
« Last Edit: May 30, 2023, 02:01:31 pm by Warfley »

Key-Real

  • Sr. Member
  • ****
  • Posts: 372
Re: Hole punching - Step by Step - with Synapse
« Reply #5 on: May 30, 2023, 02:11:10 pm »
ok, at the first step I created a server on witch I register Alice IP+Port (line 3)

why on Your Diagram in line 5 You write "request (Alice)? Should I not get the IP+Port of Bob like in step1?
I would have bough addresses, so I can do line 7.

How would line 9 looks in code?

Warfley

  • Hero Member
  • *****
  • Posts: 1745
Re: Hole punching - Step by Step - with Synapse
« Reply #6 on: May 30, 2023, 02:28:15 pm »
In this scenario Alice is a kind of server waiting for a connection request. Bob then wants to connect to alice.
The important part is that alice needs the address (IP + Port) of Bob and Bob needs the address (IP + Port) from alice.

Then both parties must send at least one punching message, which will probably be dropped by the other parties NAT. After these two messages they can communicate.

This may look like this (pseudo code, because I don't know how synapse works from the top of my head):
Alice:
Code: Pascal  [Select][+][-]
  1. socket.bind('0.0.0.0', LocalPort);
  2. repeat
  3.   socket.sendTo('register: alice', ServerAddr);
  4.   response := socket.recvFrom(ServerAddr, Timeout);
  5. until not response.DidTimeout; // Until we get the address from the server
  6.  
  7. BobAddr := response.Contents; // get the address of bob that the server send us
  8.  
  9. repeat
  10.   socket.sendTo('connect: alice', BobAddr); // functions as hole punch method
  11.   BobMessage := socket.recvFrom(BobAddr, Timeout);
  12. until not BobMessage.DidTimeout;
  13.  
  14. // We now have received a message from bob, so we know that he can receive messages from us
  15.  
Bob:
Code: Pascal  [Select][+][-]
  1. socket.bind('0.0.0.0', LocalPort);
  2. repeat
  3.   socket.sendTo('request: alice from bob', ServerAddr);
  4.   response := socket.recvFrom(ServerAddr, Timeout);
  5. until not response.DidTimeout; // Until we get the address from the server
  6.  
  7. AliceAddr := response.Contents; // get the address of bob that the server send us
  8.  
  9. repeat
  10.   socket.sendTo('connect: bob', BobAddr); // functions as hole punch method
  11.   AliceMessage := socket.recvFrom(BobAddr, Timeout);
  12. until not AliceMessage.DidTimeout;
  13.  
  14. // We now have received a message from alice, so we know that he can receive messages from us
  15.  
Server:
Code: Pascal  [Select][+][-]
  1. socket.bind('0.0.0.0', LocalPort);
  2. repeat
  3.   Message := socket.recvFrom(Any);
  4.   if Message.isRegister then
  5.     RegisteredClients[Message.ClientID] := Message.Addr
  6.   else if Message.IsRequest then
  7.   begin
  8.     TargetAddr := RegisteredClients[Message.TargetClientID];
  9.     socket.sendTo('addr: ' + Message.ClientID + Message.Addr, TargetAddr); // send Bobs addr to alice
  10.     socket.sendTo('addr: ' + Message.TargetClientID + TargetAddr, Message.Addr); // send Alices addr to bob
  11.   end;
  12. until Stopped;
  13.  

ginoo

  • Jr. Member
  • **
  • Posts: 84
Re: Hole punching - Step by Step - with Synapse
« Reply #7 on: May 31, 2023, 08:25:00 am »
HI,
I think by the end of next month I will try to implement it too with the TCP protocol. I've never done such an implementation even though I needed it. If I find difficulties I will contact you at this post.
Thanks experts.

Warfley

  • Hero Member
  • *****
  • Posts: 1745
Re: Hole punching - Step by Step - with Synapse
« Reply #8 on: May 31, 2023, 09:46:45 am »
Because TCP is a connection based protocol, many routers track the connection state and only allow for messages of that connection. This means you can't just holepunch your way through. At least not reliable through most routers

So what you should do is do UDP hole punching and build a connection based protocol on top of it (like QUIC)

Key-Real

  • Sr. Member
  • ****
  • Posts: 372
Re: Hole punching - Step by Step - with Synapse
« Reply #9 on: May 31, 2023, 10:27:28 am »
SERVER:

Code: Pascal  [Select][+][-]
  1. {$MODE OBJFPC}{$H+}
  2. uses
  3.   cthreads, Classes, blcksock, sockets, Synautil, SysUtils, HTTPSend, synaip, Process, synamisc, nettools;
  4.  
  5. type
  6.     TclientInfo = record
  7.                     ip:string;
  8.                     port:string;
  9.                   end;
  10. var
  11.     soket : TUDPBlockSocket;                             
  12.     clientA, clientB : TclientInfo;
  13.     s : string;
  14.  
  15. procedure receive;
  16. begin
  17.     repeat
  18.         if soket.CanRead(3000) then begin
  19.             soket.RecvPacket (3000);
  20.             checkError(soket);
  21.             break;
  22.         end;    
  23.     until false;
  24.     sleep(100);
  25. end;
  26.  
  27. begin
  28.     soket:= TUDPBlockSocket.Create;
  29.     soket.bind('0.0.0.0', '1500');
  30.     checkError(soket);
  31.  
  32.     writeln('Waiting for A');
  33.     receive;
  34.     clientA.ip:= soket.GetRemoteSinIP;
  35.     clientA.port:= inttostr(soket.GetRemoteSinPort);
  36.     writeln('Client A at ', clientA.ip, ':', clientA.port);    
  37.  
  38.    
  39.     writeln('Waiting for B');
  40.     receive;
  41.     clientB.ip:= soket.GetRemoteSinIP;
  42.     clientB.port:= inttostr(soket.GetRemoteSinPort);
  43.     writeln('Client B at ', clientB.ip, ':', clientB.port);    
  44.  
  45.  
  46.     writeln('send to clientA @', clientA.ip, ':', clientA.port);
  47.     soket.connect(clientA.ip, clientA.port);
  48.     s:=clientB.ip + ':' + clientB.port;
  49.     soket.SendString(s);
  50.    
  51.     writeln('send to clientB @', clientB.ip, ':', clientB.port);
  52.     soket.connect(clientB.ip, clientB.port);
  53.     s:=clientA.ip + ':' + clientA.port;
  54.     soket.SendString(s);
  55.  
  56.  
  57.     soket.CloseSocket;
  58.     soket.Free;
  59. end.
  60.  


CLIENT:

Code: Pascal  [Select][+][-]
  1. {$MODE OBJFPC}
  2. uses
  3.   cthreads, Classes, blcksock, sockets, Synautil, SysUtils, nettools;
  4.  
  5. var
  6.     soket : TUDPBlockSocket;
  7.  
  8.     clientNumber : string;
  9.     s : string;
  10.  
  11.     remoteIP, remotePort : string;
  12. begin
  13.   soket:= TUDPBlockSocket.Create;
  14.   soket.bind('0.0.0.0', '9090');
  15.   soket.connect('178.254.3.95', '1500');
  16.   checkError(soket);
  17.  
  18.  
  19.   clientNumber:= '1';
  20.   if paramcount = 1 then clientNumber:= paramstr(1);        // start with parameter    '2' for Client B
  21.  
  22.   soket.SendString(clientNumber);
  23.  
  24.   repeat
  25.     if soket.CanRead(3000) then begin
  26.       s:= soket.RecvPacket (30000);
  27.       break;
  28.     end;
  29.     sleep(100);
  30.   until false;
  31.  
  32.   remoteIP:= copy(s, 1, pos(':',s) - 1);
  33.   remotePort:= copy(s, pos(':',s) + 1, length(s));
  34.  
  35.   writeln('punching ', remoteIP, ':', remotePort);
  36.  
  37. //
  38. //
  39. // it works till here
  40. //
  41. //
  42.  
  43.   soket.connect(remoteIP, remotePort);
  44.   checkError(soket);
  45.   repeat
  46.     soket.SendString('connect from ' + clientNumber);
  47.     if soket.CanRead(3000) then begin
  48.       s:= soket.recvPacket(3000);
  49.       writeln(s);
  50.       break;
  51.     end;
  52.   until false;
  53.  
  54.   writeln('connected');
  55.  
  56.   Soket.CloseSocket;
  57.   Soket.Free;
  58. end.
  59.  



I translated Your pseudo code to synapse.
I'm getting the addresses of the client the right way.

Can't punch the hole :(


Can You pls correct the code?
« Last Edit: May 31, 2023, 04:37:46 pm by Key-Real »

ginoo

  • Jr. Member
  • **
  • Posts: 84
Re: Hole punching - Step by Step - with Synapse
« Reply #10 on: May 31, 2023, 04:25:22 pm »
Hello @Key-Real,

unfortunately now I don't have time to try, later in a few weeks I'll try too. It is not certain that it will succeed, but we hope ...

Key-Real

  • Sr. Member
  • ****
  • Posts: 372
Re: Hole punching - Step by Step - with Synapse
« Reply #11 on: June 01, 2023, 12:19:58 am »
@Warfley:

ok as I understand You wanna this:
(one way punch first)

ALLICE:

Code: Pascal  [Select][+][-]
  1. {$MODE OBJFPC}
  2. uses
  3.     blcksock, sockets, SysUtils, nettools;
  4. var
  5.     soket : TUDPBlockSocket;
  6.     s : string;
  7.     remoteIP, remotePort : string;
  8. begin
  9.  
  10.     writeln('Send to Server...');
  11.     soket:= TUDPBlockSocket.Create;
  12.     soket.bind('0.0.0.0', '9090');
  13.     soket.connect('178.254.3.95', '1500');
  14.     checkError(soket);
  15.     soket.SendString('Allice');
  16.     soket.Free;
  17.  
  18.  
  19.  
  20.     writeln('Receive from Server...');
  21.     soket:= TUDPBlockSocket.Create;
  22.     soket.bind('0.0.0.0', '9090');
  23.     repeat
  24.       if soket.CanRead(3000) then begin
  25.         s:= soket.RecvPacket (3000);
  26.         break;
  27.       end;
  28.       sleep(100);
  29.     until false;
  30.     remoteIP:= copy(s, 1, pos(':', s) - 1);
  31.     remotePort:= copy(s, pos(':', s) + 1, length(s));
  32.     soket.Free;
  33.  
  34.  
  35.  
  36.     writeln('Punching ', remoteIP, ':', remotePort, '...');
  37.     soket:= TUDPBlockSocket.Create;
  38.     soket.bind('0.0.0.0', '9090');
  39.     repeat
  40.       soket.connect(remoteIP, remotePort);
  41.       checkError(soket);
  42.  
  43.       soket.SendString('from Allice');
  44.       sleep(100);
  45.     until false;
  46.  
  47.     Soket.Free;
  48. end.
  49.  


BOB:

Code: Pascal  [Select][+][-]
  1. {$MODE OBJFPC}
  2. uses
  3.     blcksock, sockets, sysutils, nettools;
  4.  
  5. var
  6.     soket : TUDPBlockSocket;
  7.  
  8.     s : string;
  9. begin
  10.  
  11.     writeln('Send to Server...');
  12.     soket:= TUDPBlockSocket.Create;
  13.     soket.bind('0.0.0.0', '9090');
  14.     soket.connect('178.254.3.95', '1500');
  15.     checkError(soket);
  16.  
  17.     soket.SendString('Bob');
  18.    
  19.     soket.Free;
  20.  
  21.  
  22.  
  23.     writeln('Wait from Allice...');
  24.     soket:= TUDPBlockSocket.Create;
  25.     soket.bind('0.0.0.0', '9090');
  26.     repeat
  27.       if soket.CanRead(3000) then begin
  28.         s:= soket.RecvPacket (3000);
  29.         break;
  30.       end;
  31.       sleep(100);
  32.     until false;
  33.  
  34.     writeln(s);
  35.  
  36.  
  37.   Soket.Free;
  38. end.
  39.  

SERVER:

Code: Pascal  [Select][+][-]
  1. {$MODE OBJFPC}{$H+}
  2. uses
  3.     blcksock, sockets, Synautil, SysUtils, HTTPSend, synaip, Process, synamisc, nettools, crt;
  4.  
  5. type
  6.     TclientInfo = record
  7.                     ip:string;
  8.                     port:string;
  9.                   end;
  10. var
  11.     soket : TUDPBlockSocket;                             
  12.     clientAllice, clientBob : TclientInfo;
  13.     s : string;
  14.  
  15. begin
  16.    
  17.     soket:= TUDPBlockSocket.Create;
  18.     soket.bind('0.0.0.0', '1500');
  19.     checkError(soket);
  20.  
  21.     writeln('Waiting for Allice...');
  22.     repeat s:= soket.RecvPacket(3000); until s = 'Allice';
  23.     clientAllice.ip:= soket.GetRemoteSinIP;
  24.     clientAllice.port:= inttostr(soket.GetRemoteSinPort);
  25.     writeln('Allice at ', clientAllice.ip, ':', clientAllice.port);  
  26.  
  27.     writeln('Waiting for Bob...');
  28.     repeat s:= soket.RecvPacket(3000); until s = 'Bob';
  29.     clientBob.ip:= soket.GetRemoteSinIP;
  30.     clientBob.port:= inttostr(soket.GetRemoteSinPort);
  31.     writeln('Bob at ', clientBob.ip, ':', clientBob.port);  
  32.  
  33.     soket.Free;
  34.  
  35.  
  36.     soket:= TUDPBlockSocket.Create;
  37.     soket.bind('0.0.0.0', '1500');
  38.  
  39.     writeln('send to Allice ', clientBob.ip, ':', clientBob.port);
  40.     soket.connect(clientAllice.ip, clientAllice.port);
  41.     s:=clientBob.ip + ':' + clientBob.port;
  42.     soket.SendString(s);
  43. {
  44.     writeln('send to Bob ', clientAllice.ip, ':', clientAllice.port);
  45.     soket.connect(clientBob.ip, clientBob.port);
  46.     s:=clientAllice.ip + ':' + clientAllice.port;
  47.     soket.SendString(s);
  48. }
  49.     readln;
  50.     soket.Free;
  51. end.
  52.  


YOU WRONG!!!!!!!!

this can't work because in line 25 of BOB i do a bind and it gets no connections from outside!

pls write back

Warfley

  • Hero Member
  • *****
  • Posts: 1745
Re: Hole punching - Step by Step - with Synapse
« Reply #12 on: June 01, 2023, 12:48:16 am »
Try sending more packages from both Alice and Bob until you get through (the timeout of 3 seconds is way to high, make it a few Ms or so in your punching loop).

Also note that both parties, Alice and Bob need to punch a hole if both are behind a nat. And as a last note hole punching does not work with all nats in all configurations, e.g. when your punching message is discarded by the other nat, it may send an ICMP response, which most nats ignore, but some may use this event to close the punched hole. In those cases you just need to try a few times in rapid succession to trying to send messages on both sides that make it through in the small window of time the hole is open (as soon as both have received a message the hole will stay open).

The problem is all routers have different implementations of their nats, and what works with one may not work with another. After all hole punching is a hack, and not really intended behavior.

currently I have no setup to test it myself, but here is an example C code that I know from a few years ago works ok: https://github.com/mwarning/UDP-hole-punching-examples/tree/master

As you can see it just does register itself at the server, and then when the server sends the address of a new peer it just spams a few messages until the connection can be established.

You should also be able to translate this quite easily to pascal, either using the sockets u it (which provides the same sockets API used by the C code, so you can basically copy it's flow 1-1), or use the more abstracted synapse classes and methods

Key-Real

  • Sr. Member
  • ****
  • Posts: 372
Re: Hole punching - Step by Step - with Synapse
« Reply #13 on: June 01, 2023, 11:54:54 am »
This doesn't work :(
https://github.com/mwarning/UDP-hole-punching-examples/tree/master

I compiled it with GCC on both machines.
It connects to the server and get the addresses but can't receive messages from each other.

I mean Skype can do this / Games do this, why we can't?

rvk

  • Hero Member
  • *****
  • Posts: 6574
Re: Hole punching - Step by Step - with Synapse
« Reply #14 on: June 01, 2023, 12:40:22 pm »
I mean Skype can do this / Games do this, why we can't?
Are you sure Skype (and those games) really make a direct connection with each other (in your situation)?
It could be that an intermediate server is used (if hole punching fails, Skype will use a central server).

don't think I haven't look at the discussion at
https://forum.lazarus.freepascal.org/index.php/topic,52110.45.html
they tell just do

a socket.connect(remoteIP, remotePort);

this doesn't work :(
I was part of that discussion. I'm sure that's not the only thing function which was called.
Did you try those examples? For us it worked.
Although it was not my code and it has some clutter which you wouldn't need.
But did you test it?

This doesn't work :(
https://github.com/mwarning/UDP-hole-punching-examples/tree/master
I compiled it with GCC on both machines.
It connects to the server and get the addresses but can't receive messages from each other.
It says in the README.md there:
Quote
The examples have not been successfully tested and are probably broken!

 

TinyPortal © 2005-2018