Recent

Author Topic: [SOLVED] Hole punching - Step by Step - with simple example  (Read 6033 times)

Key-Real

  • Sr. Member
  • ****
  • Posts: 372
Re: Hole punching - Step by Step - with Synapse
« Reply #15 on: June 01, 2023, 01:18:32 pm »
@RVK

Should the Lazarus example of Yours write "recv[ip:port]" when You press the "Start P2P" Button?

It does not!

SERVER LOG:

unching_controller
UDP Hole Punching Controller
MAX_CONNECTIONS: 100
MAX_INACTIVE_MS: 30000
SOURCE_PORT: 1985
PWD:
---- Start print ----
--------------------------------
3r2SSU 1-6-23 13:24:05
--------------------------------
---- End print ----
---- Start print ----
--------------------------------
3r2SSU 1-6-23 13:24:22
--------------------------------
--------------------------------
BE4m0k 1-6-23 13:24:22
--------------------------------
---- End print ----
---- Start print ----
--------------------------------
3r2SSU 1-6-23 13:24:25
      B --> 84.46.70.42:13512
--------------------------------
--------------------------------
BE4m0k 1-6-23 13:24:25
--------------------------------
---- End print ----
---- Start print ----
--------------------------------
3r2SSU 1-6-23 13:24:28
      B --> 84.46.70.42:13512
--------------------------------
--------------------------------
BE4m0k 1-6-23 13:24:28
      B --> 185.237.102.35:35433
--------------------------------
---- End print ----


« Last Edit: June 01, 2023, 01:26:14 pm by Key-Real »

Key-Real

  • Sr. Member
  • ****
  • Posts: 372
Re: Hole punching - Step by Step - with Synapse
« Reply #16 on: June 01, 2023, 02:24:28 pm »
Quote from WikiPedia:




VoIP products, online gaming applications, and P2P networking software all use hole punching.

     - Telephony software Skype uses hole punching to allow users to communicate with one or more users audibly.
     - Fast-paced online multi-player games may use a hole punching technique or require users to create a permanent firewall pinhole in order to reduce network latency.
     - VPN applications such as Hamachi, ZeroTier, and Tailscale utilize hole punching to allow users to connect directly to subscribed devices behind firewalls.
     - Decentralized peer-to-peer file sharing software relies on hole punching for file distribution.



So what they do, what we do wrong?

Warfley

  • Hero Member
  • *****
  • Posts: 1758
Re: Hole punching - Step by Step - with Synapse
« Reply #17 on: June 01, 2023, 03:04:20 pm »
It says in the README.md there:
Quote
The examples have not been successfully tested and are probably broken!

Back in Uni I used this as a "guide" to create some working hole punching code in C++. I don't have that code anymore, otherwise I would have posted it, but in general this can work.

So what they do, what we do wrong?
But that said, as I already wrote previously, different routers behave differently, and hole punching is easier in some networks than others. It's a hack. This SO post summarizes it quite well: https://stackoverflow.com/a/6794529

You can try different sending patterns, e.g. send in bursts with some pauses in between, which seems to help sometimes

rvk

  • Hero Member
  • *****
  • Posts: 6584
Re: Hole punching - Step by Step - with Synapse
« Reply #18 on: June 01, 2023, 04:17:33 pm »
I have some python code which I used to test if punching a whole was really possible in my situation.
Converted to python3 (original code was here https://gist.github.com/somic/224795#file-udp_hole_punch_tester-py)

For me that still worked. I'll need to look at the other code here as to why that doesn't work.

Code: Python  [Select][+][-]
  1. #!/usr/bin/env python
  2. #
  3. # udp_hole_punch_tester.py - UDP Hole Punching test tool
  4. #
  5. # Usage: udp_hole_punch_tester.py remote_host remote_port
  6. #
  7. # Run this script simultaneously on 2 hosts to test if they can punch
  8. # a UDP hole to each other.
  9. #
  10. # * remote_port should be identical on 2 hosts.
  11. # * if remote_port < 1024, must be root.
  12. # * tested on python 2.5.
  13. #
  14. # Copyright (C) 2009 Dmitriy Samovskiy, http://somic.org
  15. #
  16. # License: Apache License, Version 2.0
  17. #          http://www.apache.org/licenses/
  18. #
  19.  
  20. import sys, os, time, socket, random
  21. from select import select
  22.  
  23. def log(*args):
  24.     print(time.asctime(), ' '.join([str(x) for x in args]))
  25.  
  26. def puncher(remote_host, port):
  27.     sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
  28.     sock.bind(('', port))
  29.  
  30.     my_token = str(random.random())
  31.     log("my_token =", my_token)
  32.     remote_token = "_"
  33.  
  34.     sock.setblocking(0)
  35.     sock.settimeout(5)
  36.  
  37.     remote_knows_our_token = False
  38.  
  39.     for i in range(60):
  40.         r,w,x = select([sock], [sock], [], 0)
  41.  
  42.         if remote_token != "_" and remote_knows_our_token:
  43.             log("we are done - hole was punched from both ends")
  44.             break
  45.  
  46.         if r:
  47.             data, addr = sock.recvfrom(1024)
  48.             log("recv:", data)
  49.             if remote_token == "_":
  50.                 remote_token = data.split()[0]
  51.                 log("remote_token is now", remote_token)
  52.             if len(data.split()) == 3:
  53.                 log("remote end signals it knows our token")
  54.                 remote_knows_our_token = True
  55.  
  56.         if w:
  57.             data = "%s %s" % (my_token, remote_token)
  58.             if remote_token != "_": data += " ok"
  59.             log("sending:", data)
  60.             dd = bytes(data, 'utf-8')
  61.             sock.sendto(dd, (remote_host, port))
  62.             log("sent", i)
  63.         time.sleep(0.5)
  64.  
  65.     log("done")
  66.     sock.close()
  67.  
  68.     return remote_token != "_"
  69.  
  70. if __name__ == '__main__':
  71.     remote_host = sys.argv[1]
  72.     port = int(sys.argv[2])
  73.  
  74.     if puncher(remote_host, port):
  75.         log("Punched UDP hole to %s:%d successfully" % (remote_host, port))
  76.     else:
  77.         log("Failed to punch hole")

rvk

  • Hero Member
  • *****
  • Posts: 6584
Re: Hole punching - Step by Step - with Synapse
« Reply #19 on: June 01, 2023, 04:43:23 pm »
For me that still worked. I'll need to look at the other code here as to why that doesn't work.
What I find weird about this code is that it works without any port translation.
I even tried it on port 1500 UDP which I have redirected to another computer.
And then it still works.

So just continuously pushing a string to the other side should be enough.

Edit: Ok, that only works if you are really on two different locations.
If you are on the same location and use your external address the NAT kicks in and translates the port and it doesn't work with the same port anymore.
« Last Edit: June 01, 2023, 04:57:19 pm by rvk »

Key-Real

  • Sr. Member
  • ****
  • Posts: 372
Re: Hole punching - Step by Step - with Synapse
« Reply #20 on: June 01, 2023, 06:12:20 pm »
I translated the
https://github.com/mwarning/UDP-hole-punching-examples/tree/master

GOOD and BAD NEWS!

the good is it works :)

the Bad is not everywhere :(

today morning I tried this the whole time at work. I opened one client on the works internet connection and the other through my mobile phone.
the server gets the messages from them, but the clients are unable to connect to each other.
so I tried this at home. I had the same scenario. So I called a friend to open a client on his side and I open one on my home machine trough my internet provider. And look, it works. So I did this again but now through my mobile. Hire is the bad news. I can connect to my friends client but he can't to my mobile :(

I mark this Topic as solved.
here the code:

SERVER:

Code: Pascal  [Select][+][-]
  1. {$MODE OBJFPC}{$H+}
  2. uses
  3.     sockets, SysUtils, crt;
  4.  
  5. function inet_ntoa(addr:in_addr):Pchar;cdecl;external 'libc' name 'inet_ntoa';
  6.  
  7. procedure diep(s:string);
  8. begin
  9.     writeln(s);
  10.     halt(1);
  11. end;
  12.  
  13. type
  14.      Tclient = record
  15.                 host : dword;
  16.                 port : word;
  17.                end;
  18.  
  19. var
  20.     si_me, si_other : sockaddr_in;
  21.     s, i, j, slen : dword;
  22.     buf : array [0..512] of char;
  23.     clients : array [0..10] of Tclient; // 10 clients. Notice that we're not doing any bound checking.
  24.     n : longint;
  25.    
  26. begin
  27.     slen:= sizeof(si_other);
  28.  
  29.     // Create a UDP socket
  30.     s:= fpsocket(AF_INET, SOCK_DGRAM, IPPROTO_UDP);
  31.     if s = -1 then diep('socket');
  32.  
  33.  
  34.     // si_me stores our local endpoint. Remember that this program
  35.     // has to be run in a network with UDP endpoint previously known
  36.     // and directly accessible by all clients. In simpler terms, the
  37.     // server cannot be behind a NAT.
  38.     fillchar(si_me, 0, sizeof(si_me));
  39.     si_me.sin_family:= AF_INET;
  40.     si_me.sin_port:= htons(9930);
  41.     si_me.sin_addr.s_addr:= htonl(INADDR_ANY);
  42.     if fpbind(s, @si_me, sizeof(si_me)) = -1 then diep('bind');
  43.    
  44.     while n < 2 do begin
  45.    
  46.         // When a new client sends a datagram...
  47.         if fprecvfrom(s, @buf, 512, 0, @si_other, @slen) = -1 then diep('recvfrom');
  48.         // The client's public UDP endpoint data is now in si_other.
  49.         // Notice that we're completely ignoring the datagram payload.
  50.         // If we want to support multiple clients inside the same NAT,
  51.         // we'd have clients send their own private UDP endpoints
  52.         // encoded in some way inside the payload, and store those as
  53.         // well.
  54.         writeln('Received packet from ', inet_ntoa(si_other.sin_addr), ':', ntohs(si_other.sin_port));
  55.         // Now we add the client's UDP endpoint in our list.
  56.         clients[n].host:= si_other.sin_addr.s_addr;
  57.         clients[n].port:= si_other.sin_port;
  58.         inc(n);
  59.         // And then tell everybody about everybody's public UDP endpoints
  60.         for i:=0 to n - 1 do begin        
  61.             si_other.sin_addr.s_addr:= clients[i].host;
  62.             si_other.sin_port:= clients[i].port;
  63.             // We send a datagram for each client in our list. Of course,
  64.             // we could also assemble a single datagram and send that.
  65.             for j:= 0 to n - 1 do begin
  66.                 // The payload is the client's public UDP endpoint, clients[j]
  67.                 writeln('Sending to ', inet_ntoa(si_other.sin_addr), ':', ntohs(si_other.sin_port));
  68.                 // We're sending binary data here, using the server's byte order.
  69.                 // In your code, you should make sure every client agrees on the endianness.
  70.                 if fpsendto(s, @clients[j], 6, 0, @si_other, slen) = -1 then diep('sendto');
  71.             end;
  72.         end;
  73.         writeln('Now we have ', n, ' clients');
  74.         // And we go back to listening. Notice that since UDP has no notion
  75.         // of connections, we can use the same socket to listen for data
  76.         // from different clients.
  77.     end;
  78.  
  79.     // Actually, we never reach this point...
  80.     closeSocket(s);
  81. end.
  82.  


CLIENT:

Code: Pascal  [Select][+][-]
  1. {$MODE OBJFPC}
  2. uses
  3.     sockets, SysUtils
  4. {$IFDEF WINDOWS}
  5.     , winsock
  6. {$ENDIF}
  7.     ;
  8.  
  9.  
  10. {$IFNDEF WINDOWS}
  11. function inet_aton(cp:Pchar; addr:Pin_addr):longint;cdecl;external 'libc' name 'inet_aton';
  12. function inet_ntoa(addr:in_addr):Pchar;cdecl;external 'libc' name 'inet_ntoa';
  13. {$ENDIF}
  14.  
  15.  
  16. const SRV_IP = '178.254.3.95';
  17.  
  18.  
  19. procedure diep(s:string);
  20. begin
  21.     writeln(s);
  22.     halt(1);
  23. end;
  24.  
  25. type
  26.          Tclient = record
  27.                         host : dword;
  28.                         port : word;
  29.                    end;
  30.  
  31. var
  32.         si_me, si_other : sockaddr_in;
  33.        
  34.     s, i, f, j, k, slen : dword;
  35.     buf, server : Tclient;
  36.     peers : array [0..10] of Tclient;
  37.     n : longint;
  38. begin
  39.  
  40.         slen:= sizeof(si_other);
  41.  
  42.         s:= fpsocket(AF_INET, SOCK_DGRAM, IPPROTO_UDP);
  43.         if s = -1 then diep('socket');
  44.  
  45.  // Our own endpoint data
  46.     fillchar(si_me, 0, sizeof(si_me));
  47.     si_me.sin_family:= AF_INET;
  48.     si_me.sin_port:= htons(9930); // This is not really necessary, we can also use 0 (any port)
  49.     si_me.sin_addr.s_addr:= htonl(INADDR_ANY);
  50.  
  51.  
  52.     // The server's endpoint data
  53.     fillchar(si_other, 0, sizeof(si_other));
  54.     si_other.sin_family:= AF_INET;
  55.     si_other.sin_port:= htons(9930);
  56. {$IFDEF WINDOWS}
  57.     si_other.sin_addr.s_addr:=inet_addr(SRV_IP);
  58. {$ELSE}
  59.     if inet_aton(SRV_IP, @si_other.sin_addr) = 0 then diep('aton');
  60. {$ENDIF}
  61.  
  62.     // Store the server's endpoint data so we can easily discriminate between server and peer datagrams.
  63.     server.host:= si_other.sin_addr.s_addr;
  64.     server.port:= si_other.sin_port;
  65.  
  66.         // Send a simple datagram to the server to let it know of our public UDP endpoint.
  67.     // Not only the server, but other clients will send their data through this endpoint.
  68.     // The datagram payload is irrelevant, but if we wanted to support multiple
  69.     // clients behind the same NAT, we'd send our won private UDP endpoint information
  70.     // as well.
  71.     if fpsendto(s, pchar('hi'), 2, 0, @si_other, slen) = -1 then diep('sendto');
  72.  
  73.  
  74.  
  75.  
  76.     // Right here, our NAT should have a session entry between our host and the server.
  77.     // We can only hope our NAT maps the same public endpoint (both host and port) when we
  78.     // send datagrams to other clients using our same private endpoint.
  79.     while true do begin
  80.    
  81.         // Receive data from the socket. Notice that we use the same socket for server and
  82.         // peer communications. We discriminate by using the remote host endpoint data, but
  83.         // remember that IP addresses are easily spoofed (actually, that's what the NAT is
  84.         // doing), so remember to do some kind of validation in here.
  85.         if fprecvfrom(s, @buf, sizeof(buf), 0, @si_other, @slen) = -1 then diep('recvfrom');
  86.  
  87.         writeln('Received packet from ', inet_ntoa(si_other.sin_addr), ':', ntohs(si_other.sin_port));
  88.         if (server.host = si_other.sin_addr.s_addr) and (server.port = si_other.sin_port) then begin
  89.        
  90.             // The datagram came from the server. The server code is set to send us a
  91.             // datagram for each peer, in which the payload contains the peer's UDP
  92.             // endpoint data. We're receiving binary data here, sent using the server's
  93.             // byte ordering. We should make sure we agree on the endianness in any
  94.             // serious code.
  95.             f:= 0;
  96.             // Now we just have to add the reported peer into our peer list
  97.             for i:= 0 to n do begin
  98.            
  99.                 if (peers[i].host = buf.host) and (peers[i].port = buf.port) then begin
  100.                
  101.                     f:= 1;
  102.                 end;
  103.             end;
  104.             // Only add it if we didn't have it before.
  105.             if f = 0 then begin
  106.                 peers[n].host:= buf.host;
  107.                 peers[n].port:= buf.port;
  108.                 inc(n);
  109.             end;
  110.             si_other.sin_addr.s_addr:= buf.host;
  111.             si_other.sin_port:= buf.port;
  112.             writeln('Added peer ', inet_ntoa(si_other.sin_addr), ':', ntohs(si_other.sin_port));
  113.             writeln('Now we have ', n, ' peers');
  114.             // And here is where the actual hole punching happens. We are going to send
  115.             // a bunch of datagrams to each peer. Since we're using the same socket we
  116.             // previously used to send data to the server, our local endpoint is the same
  117.             // as before.
  118.             // If the NAT maps our local endpoint to the same public endpoint
  119.             // regardless of the remote endpoint, after the first datagram we send, we
  120.             // have an open session (the hole punch) between our local endpoint and the
  121.             // peer's public endpoint. The first datagram will probably not go through
  122.             // the peer's NAT, but since UDP is stateless, there is no way for our NAT
  123.             // to know that the datagram we sent got dropped by the peer's NAT (well,
  124.             // our NAT may get an ICMP Destination Unreachable, but most NATs are
  125.             // configured to simply discard them) but when the peer sends us a datagram,
  126.             // it will pass through the hole punch into our local endpoint.
  127.             for k:=0 to 10 - 1 do begin
  128.                 // Send 10 datagrams.
  129.                 for i:=0 to n - 1  do begin
  130.                
  131.                     si_other.sin_addr.s_addr := peers[i].host;
  132.                     si_other.sin_port := peers[i].port;
  133.                     // Once again, the payload is irrelevant. Feel free to send your VoIP
  134.                     // data in here.
  135.  
  136.                     writeln('sending to ', inet_ntoa(si_other.sin_addr), ':', ntohs(si_other.sin_port ));
  137.  
  138.                     if fpsendto(s, pchar('hi'), 2, 0, @si_other, slen) = -1 then diep('sendto()');
  139.                 end;
  140.             end;
  141.  
  142.             writeln('packages send');
  143.  
  144.         end else begin
  145.        
  146.             // The datagram came from a peer
  147.             for i:= 0 to n - 1 do begin
  148.            
  149.                 // Identify which peer it came from
  150.                 //if (peers[i].host = buf.host) and (peers[i].port = buf.port) then begin                
  151.                     // And do something useful with the received payload
  152.                     writeln('Received from peer ', i);
  153.                 //    break;
  154.                // end;
  155.             end;
  156.  
  157.             // It is possible to get data from an unregistered peer. These are some reasons
  158.             // I quickly came up with, in which this can happen:
  159.             // 1. The server's datagram notifying us with the peer's address got lost,
  160.             //    or it hasn't arrived yet (likely)
  161.             // 2. A malicious (or clueless) user is sending you data on this endpoint (maybe
  162.             //    unlikely)
  163.             // 3. The peer's public endpoint changed either because the session timed out,
  164.             //    or because its NAT did not assign the same public endpoint when sending
  165.             //    datagrams to different remote endpoints. If this happens, and we're able
  166.             //    to detect this situation, we could change our peer's endpoint data to
  167.             //    the correct one. If we manage to pull this off correctly, even if at most
  168.             //    one client has a NAT that doesn't support hole punching, we can communicate
  169.             //    directly between both peers.
  170.         end;
  171.     end;
  172.  
  173.     // Actually, we never reach this point...
  174.     closeSocket(s);
  175. end.
  176.  

Key-Real

  • Sr. Member
  • ****
  • Posts: 372
Re: [SOLVED] Hole punching - Step by Step - with simple example
« Reply #21 on: June 01, 2023, 06:14:04 pm »
btw.

Ideas what is the background behind this issue?

Is it possible to solve?

rvk

  • Hero Member
  • *****
  • Posts: 6584
Re: [SOLVED] Hole punching - Step by Step - with simple example
« Reply #22 on: June 01, 2023, 06:28:29 pm »
Ideas what is the background behind this issue?
Method of NAT implementation.

Something like this:
Quote
UDP hole punching will not work with symmetric NAT devices (also known as bi-directional NAT) which tend to be found in large corporate networks. In symmetric NAT, the NAT's mapping associated with the connection to the known STUN server is restricted to receiving data from the known server, and therefore the NAT mapping the known server sees is not useful information to the endpoint.
https://en.wikipedia.org/wiki/UDP_hole_punching

So with a stricter NAT, only successful connections are registered as allowed connections.
And since client1 and client2 never establish a connection to each other, they can't get through.

Warfley

  • Hero Member
  • *****
  • Posts: 1758
Re: Hole punching - Step by Step - with Synapse
« Reply #23 on: June 01, 2023, 07:56:17 pm »
the server gets the messages from them, but the clients are unable to connect to each other.
so I tried this at home. I had the same scenario. So I called a friend to open a client on his side and I open one on my home machine trough my internet provider. And look, it works. So I did this again but now through my mobile. Hire is the bad news. I can connect to my friends client but he can't to my mobile :(
Have you checked if all the ports and addresses are correct? Because one of the main problems with network programming is actually byte order (which as stated in the comments is ignored in this example). If your phone and your Desktop have different byte orders, they may interpret the data differently. So you should always use network byte order to send data and convert it upon receival. The ntohl function converts a long (i.e. 32 bit) integer from network byte order to host byte order. htonl is the opposite converting host to network.

Also, just to note, you are using inet_ntoa which is basically the C version of IntToStr for inet addresses. You can just use the function NetAddrToStr (HostAddrToStr for Host byte order, but socket addresses are in network byte order) instead.

Lastly do you use a wifi with you phone or mobile network? Because Mobile Networks are quite comple. Basically in order for you not to change your IP address every time you connect to a new tower, the datacenters at each station that doe the tunneling of data, are connected through one giant NAT network, a so called Large Scale Nat. They have higher security standards than your usual home router NATs and are usually symmetric NATs.

With google I have found this link: https://drive.google.com/file/d/0B1IimJ20gG0SY2NvaE4wRVVMbG8/view?resourcekey=0-C9_6uLzao8eCEwa1qRhv7w as well as this paper behind a paywall (which may or may not be circumventable via scihub): http://dx.doi.org/10.7125/APAN.30.5

 

TinyPortal © 2005-2018