I had used simultaneous-open TCP punching as a fallback for UDP punching when I was doing [0] and it did help in 10-20% of cases when (a fairly elaborate version of) UDP punching failed. That's on a scale of several hundred thousand mediated connections per day.
One caveat though is that it requires implementing a bot-like functionality in the clients, meaning that a mediating server should be able to tell a client - "create a socket, bind it to this ip:port, wait, wait... connect to that ip:port". Obviously, this is an ideal platform for DDoS attacks if someone ever manages to re-point clients to a rogue mediation server. So, yeah, it works well, but there are some not so obvious trade-offs.
In our experience with a deployed P2P system, UDP hole punching is more successful with all the strange NAT boxes deployed. Our success rate is 95.3% now in the wild. As said above, this TCP fallback sounds like an excellent way to get to 97% connection success rate!
STUN[1] does a very similar thing, including some trickery to get through all kinds of plastic boxes messing with your packets.
AeroFS[2] is another, interesting approach. It's basically a dropbox clone where all data remains on the user's computer. Transfers are encrypted with TLS and tunneled through the company's servers.
Another interesting thing might be Opera Unite[3]. It's an in-browser web server that can be souped up using various extensions like photo sharing, games or chat. As far as I know its NAT traversal is somewhat limited, though.
While on the topic of clever UDP-based NAT traversal techniques, I think my favorite to date is pwnat [http://samy.pl/pwnat/]. Uses ICMP in a clever way, requires no 3rd party.
This is exactly what one of those UDP "weird hole punching tricks" does. Except of course without connections, it not being TCP. nat-traverse (http://m19s28.dyndns.org/iblech/nat-traverse/) works this way as I understand it.
As the author notes, for such hack to have any chances of working reliably a third party is needed to synchronize connection setup. But a third party with a public IP could also forward an encrypted TCP connection between two machines that do not have public IP addresses. Such approach does not require any hacks and gives a strong guarantee that the third party can not snoop the connection.
The only thing that the third party learns is that there is a connection, but the proposes solution also has this drawback.
Yes, but your approach requires a lot of bandwidth on the 3rd party.
But a 3rd party doing nothing except helping with synchronization uses barely any bandwidth and someone could easily host it for free on some server that is not otherwise fully used, simply as a service to the public.
There was a good thread on p2p-hackers a little while back. I apologize for not digging up the link but I'm on my way to bed. I think the thread was something like "state of the art NAT traversal"
I have to disagree on that - can you really trust encrpytion? Are you sure? On the other hand, why depend on (supposedly) strong encryption against 3rd party if you can just avoid passing the message through their hands altogether?
This seems like a great stuff if it works. Unbeleviable I was able to avoid knowing about it until now. Kudos to top poster, great info.
If you don't want to trust encryption you are screwed. Unencrypted TCP doesn't give any snooping protection, it also doesn't give any way to enforce and validate that a route that packets traverse is trusted.
This won't happen. When IPv4 dies, then so will NAT. But we'll all need IPv6 stateful firewalls at home. Generally, they'll block all incoming attempts to make new connections, and allow any outbound connection attempts.
Now you have the same problem. The same solutions will need to apply here, too. So NAT isn't really the fundamental problem here.
But a firewall I can open. heck most consumer routers came preinstalled with holes on the firewall/NAT for FTP,IRC,MSN and a bunch of long dead protocols that required the server to connect to the client...
with ipv4+nat often I can't allow a connection because i don't have the IP visible. i'd have to have the server to add my internal IP (which will surely conflict) to their routing table passing trhu my external IP on the router.
NAT'ing is such an ingrained way of doing things, and it also happens to be a great way to separate your network from the rest of the network/internet. I don't think it'll go away any time soon.
Please allow me to be pedantic here. What actually separates your network from the rest of the Internet is called a firewall. It is unfortunate that common usage has conflated the two terms to the point that many people believe they must be inside a NAT to be firewalled from the outside. It's actually one of many misunderstandings related to the IPv6 transition.
I'm not sure what OP has against UPnP port mapping. It's not an ideal option and certainly not the best-designed specification, but it's a way to explicitly tell the NAT to do what we want, rather than having to trick it into doing what we want.
It's primarily an IPv6 transition technology, but it comes with NAT traversal, hole-punching and all that built-in. What you get is a virtual IPv6 network interface to which you simply bind your socket, and you can connect to other Teredo / IPv6 sockets... if the whimsies of the Gods of Middleboxes allow.
What makes it attractive is that it is deployed on all Windows machines WinXP and later (and enabled by default on Vista and later), giving it a huge deployment base. It's not present on non-Windows machines, of course, but there is a liberally licensed implementation for OSX and Linux called Miredo. uTorrent is one popular application that uses Teredo.
However, Teredo tunneling does not work very reliably (apparently, its designers traded off connectivity for additional security), and it would be unadvisable to have that as your only NAT traversal method. But I think getting that option with minimal additional code is not a bad deal.
From my understanding a NAT is going to remap an internal port to a different external port and upon receiving a RST will delete the mapping. Is this not correct?
Also, doesnt connection reversal solve this problem? (Connection reversal: have A connect to intermediary S. have B connect to intermediary S. S sends A B's info and vice versa. A connects to B before closing connection with S.) This also does not require S to forward data, only connection information. Am I missing something there?
There are like 4 different NAT port translation models. In some of the models, the NAT algorithm will actually preserve the ephemeral port number in the NAT mapping. This falls apart quickly when you have a few NAT clients. But it can make legacy software work a lot better. We have weeded most of that software out at this point, or at least provided application gateways that inspect the content of the packet and tweak little things - consider how you need an extra kernel module for FTP and SIP and stuff.
I'm surprised this works... or at least works reliably. I understood many simple firewalls just blocked inbound SYNs (without performing any kind of IP address lookup). This would obviously prevent this mechanism from working. Is this not (or no longer) the case?
I am not an expert in this area, but from what I understand, firewalls keep a list of outbound connections. If inbound connection comes from a known destination IP+port, it will be forwarded to the internal "source" IP+port.
This is how I understand it: if both A and B are behind firewalls, they use C to reach an agreement about IPs and ports used. Then A sends a packet to B, which is silently dropped at B's firewall. Then B send a packet to A - since it looks like an "answer" to previous request it is forwarded by A's firewall to A. Then A sends another packet to B, which is also forwarded by B's firewall to B. Voila, connection made. :)
Note that this is just my understanding, so I would appreciate if someone more knowledgeable in this area would chime in.
This works and it works reasonably well.
I had used simultaneous-open TCP punching as a fallback for UDP punching when I was doing [0] and it did help in 10-20% of cases when (a fairly elaborate version of) UDP punching failed. That's on a scale of several hundred thousand mediated connections per day.
One caveat though is that it requires implementing a bot-like functionality in the clients, meaning that a mediating server should be able to tell a client - "create a socket, bind it to this ip:port, wait, wait... connect to that ip:port". Obviously, this is an ideal platform for DDoS attacks if someone ever manages to re-point clients to a rogue mediation server. So, yeah, it works well, but there are some not so obvious trade-offs.
[0] http://swapped.cc/hamachi
Ah! So that's how you guys did it! I always wondered what you were doing beyond UDP hole punching.
Is there a Free software package that does something like this today?
TCP simultaneous open is very difficult to use.
In our experience with a deployed P2P system, UDP hole punching is more successful with all the strange NAT boxes deployed. Our success rate is 95.3% now in the wild. As said above, this TCP fallback sounds like an excellent way to get to 97% connection success rate!
Note that there is an upcoming IETF Internet Standard describing UDP hole punching: http://tools.ietf.org/html/draft-ietf-ppsp-peer-protocol-06#...
We've implemented this as LGPL code: svn.tribler.org/libswift/branches/ppsp-03/
STUN[1] does a very similar thing, including some trickery to get through all kinds of plastic boxes messing with your packets.
AeroFS[2] is another, interesting approach. It's basically a dropbox clone where all data remains on the user's computer. Transfers are encrypted with TLS and tunneled through the company's servers.
Another interesting thing might be Opera Unite[3]. It's an in-browser web server that can be souped up using various extensions like photo sharing, games or chat. As far as I know its NAT traversal is somewhat limited, though.
[1] http://en.wikipedia.org/wiki/Session_Traversal_Utilities_for... [2] https://aerofs.com/ [3] http://unite.opera.com/applications/
While on the topic of clever UDP-based NAT traversal techniques, I think my favorite to date is pwnat [http://samy.pl/pwnat/]. Uses ICMP in a clever way, requires no 3rd party.
Note that STUN itself does not do traversal for you - it simply gives you enough information to do it yourself.
TCP/IP Illustrated Vol. 1, Stevens, pg: 250-252
This is exactly what one of those UDP "weird hole punching tricks" does. Except of course without connections, it not being TCP. nat-traverse (http://m19s28.dyndns.org/iblech/nat-traverse/) works this way as I understand it.
here's an excellent paper on hole punching/nat traversal http://www.brynosaurus.com/pub/net/p2pnat/
For anyone who wants to fiddle as well, I wrote some very simple Ruby code several years ago to do it too: http://www.rubyinside.com/skype-style-firewall-busting-with-...
As the author notes, for such hack to have any chances of working reliably a third party is needed to synchronize connection setup. But a third party with a public IP could also forward an encrypted TCP connection between two machines that do not have public IP addresses. Such approach does not require any hacks and gives a strong guarantee that the third party can not snoop the connection.
The only thing that the third party learns is that there is a connection, but the proposes solution also has this drawback.
Yes, but your approach requires a lot of bandwidth on the 3rd party.
But a 3rd party doing nothing except helping with synchronization uses barely any bandwidth and someone could easily host it for free on some server that is not otherwise fully used, simply as a service to the public.
That's standardized as TURN, if I'm not mistaken ( https://en.wikipedia.org/wiki/Traversal_Using_Relays_around_... ). Other NAT traversal mechanisms (STUN) don't make the server pay for bandwidth. (Which methods work with TCP, in practice?)
There was a good thread on p2p-hackers a little while back. I apologize for not digging up the link but I'm on my way to bed. I think the thread was something like "state of the art NAT traversal"
I have to disagree on that - can you really trust encrpytion? Are you sure? On the other hand, why depend on (supposedly) strong encryption against 3rd party if you can just avoid passing the message through their hands altogether?
This seems like a great stuff if it works. Unbeleviable I was able to avoid knowing about it until now. Kudos to top poster, great info.
If you don't want to trust encryption you are screwed. Unencrypted TCP doesn't give any snooping protection, it also doesn't give any way to enforce and validate that a route that packets traverse is trusted.
1 reply →
Because nobody has linked it yet, the referenced XKCD is http://xkcd.com/949/
Why not just adopt ipv6 and connect to the box directly and intuitively when all ipv4 nat boxes in the middle die?
This won't happen. When IPv4 dies, then so will NAT. But we'll all need IPv6 stateful firewalls at home. Generally, they'll block all incoming attempts to make new connections, and allow any outbound connection attempts.
Now you have the same problem. The same solutions will need to apply here, too. So NAT isn't really the fundamental problem here.
But a firewall I can open. heck most consumer routers came preinstalled with holes on the firewall/NAT for FTP,IRC,MSN and a bunch of long dead protocols that required the server to connect to the client...
with ipv4+nat often I can't allow a connection because i don't have the IP visible. i'd have to have the server to add my internal IP (which will surely conflict) to their routing table passing trhu my external IP on the router.
This is the problem ipv6 solves.
We have had customers ask us if we support NAT for IPv6, so don't assume NAT will automatically die.
Sadly enough.
I can kinda see edge cases where 1-1 NAT makes sense, depending on the product, or are they asking for NAPT like most people think of NAT?
1 reply →
Nature will find a way :)
your client will probably have it's market eaten by someone who understand the tech. hopefully.
/me insert faster horse fallacy
NAT'ing is such an ingrained way of doing things, and it also happens to be a great way to separate your network from the rest of the network/internet. I don't think it'll go away any time soon.
Please allow me to be pedantic here. What actually separates your network from the rest of the Internet is called a firewall. It is unfortunate that common usage has conflated the two terms to the point that many people believe they must be inside a NAT to be firewalled from the outside. It's actually one of many misunderstandings related to the IPv6 transition.
9 replies →
NAT separates your network as much as buying 10.10.10.* and assigning 10.10.10.0-100 to the internal network.
I'm not sure what OP has against UPnP port mapping. It's not an ideal option and certainly not the best-designed specification, but it's a way to explicitly tell the NAT to do what we want, rather than having to trick it into doing what we want.
Also, one option not mentioned here yet is Teredo: http://en.m.wikipedia.org/wiki/Teredo_tunneling
It's primarily an IPv6 transition technology, but it comes with NAT traversal, hole-punching and all that built-in. What you get is a virtual IPv6 network interface to which you simply bind your socket, and you can connect to other Teredo / IPv6 sockets... if the whimsies of the Gods of Middleboxes allow.
What makes it attractive is that it is deployed on all Windows machines WinXP and later (and enabled by default on Vista and later), giving it a huge deployment base. It's not present on non-Windows machines, of course, but there is a liberally licensed implementation for OSX and Linux called Miredo. uTorrent is one popular application that uses Teredo.
However, Teredo tunneling does not work very reliably (apparently, its designers traded off connectivity for additional security), and it would be unadvisable to have that as your only NAT traversal method. But I think getting that option with minimal additional code is not a bad deal.
From my understanding a NAT is going to remap an internal port to a different external port and upon receiving a RST will delete the mapping. Is this not correct?
Also, doesnt connection reversal solve this problem? (Connection reversal: have A connect to intermediary S. have B connect to intermediary S. S sends A B's info and vice versa. A connects to B before closing connection with S.) This also does not require S to forward data, only connection information. Am I missing something there?
There are like 4 different NAT port translation models. In some of the models, the NAT algorithm will actually preserve the ephemeral port number in the NAT mapping. This falls apart quickly when you have a few NAT clients. But it can make legacy software work a lot better. We have weeded most of that software out at this point, or at least provided application gateways that inspect the content of the packet and tweak little things - consider how you need an extra kernel module for FTP and SIP and stuff.
The models are detailed nicely in this diagram. https://en.wikipedia.org/wiki/Network_address_translation#Me...
http://nmap.org/misc/split-handshake.pdf
Isn't this called "NAT punching", and generally used by games and other sw to establish peer-to-peer connections?
That's usually UDP. With this method, you "get all of the utility [TCP] provides without trying to reproduce it over UDP."
I'm surprised this works... or at least works reliably. I understood many simple firewalls just blocked inbound SYNs (without performing any kind of IP address lookup). This would obviously prevent this mechanism from working. Is this not (or no longer) the case?
I am not an expert in this area, but from what I understand, firewalls keep a list of outbound connections. If inbound connection comes from a known destination IP+port, it will be forwarded to the internal "source" IP+port.
This is how I understand it: if both A and B are behind firewalls, they use C to reach an agreement about IPs and ports used. Then A sends a packet to B, which is silently dropped at B's firewall. Then B send a packet to A - since it looks like an "answer" to previous request it is forwarded by A's firewall to A. Then A sends another packet to B, which is also forwarded by B's firewall to B. Voila, connection made. :)
Note that this is just my understanding, so I would appreciate if someone more knowledgeable in this area would chime in.
BitTorrent Sync [1] can help with the explicit file sharing problem from the xkcd. Too bad it's not open source.
[1]: http://labs.bittorrent.com/experiments/sync.html
Is this the sort of technology that P2P systems like BitTorrent use? I've never understood how BT works in setups that block incoming connections.
what was the title of the book you read?