Comment by vivzkestrel
1 day ago
what did you refer for building this? did you read the protocol or did you go through other implementations? curious because it always makes me wonder how people build stuff like this from scratch
1 day ago
what did you refer for building this? did you read the protocol or did you go through other implementations? curious because it always makes me wonder how people build stuff like this from scratch
Interestingly (or maybe not) I started writing a Bittorrent client last week, also in Go. I don't really like AI/LLMs for coding, and my goal is to really learn from it. Not to just deliver some piece of code that an AI wrote...
So the first step is to figure out if there is an actual spec for the protocol. Well, in the case of Bittorrent there is a spec, but it's kinda... brief. Basically, this is the spec: https://www.bittorrent.org/beps/bep_0003.html
There are some additional extensions here: https://www.bittorrent.org/beps/bep_0000.html
It's very important to break the work up in tiny steps that each give some kind of a result. E.g. I started with parsing .torrent files, for which you have to implement bencoding.
That was interesting, because I downloaded the Arch Linux .torrent and found out it was basically incorrect: it has a url-list key which isn't mentioned in the spec. Then you have to go down a rabbit hole of some research trying to find out what the heck is up with that (turned out to be https://www.bittorrent.org/beps/bep_0019.html). Finally, was able to successfully parse the Debian Linux .torrent file.
Then came implementing the tracker Announce HTTP request and the peer protocol. The peer protocol was tricky, and that's when an experimentation mindset comes in handy. I stripped the announce URL from the Debian torrent and loaded it up in KTorrent, that way it wasn't able to find any peers. I used the "Add peer" option to add my client so I could see what messages it was sending me, and alongside I would implement sending those to KTorrent messages too. A lot of trial and error and debugging.
I have to confess that from time to time I had to ask ChatGPT about some small protocol details because they just didn't show up in the spec. There's also a lot of details that clients have implemented over time but which aren't really documented well. Mostly algorithms for which pieces to download, which peers to choose, how to "choke"/"unchoke", etc, is not well-defined. But web searches help out a lot.
Also an honorable mention to https://wiki.theory.org/Main_Page which has some nice information.
I'm now at a point where I can fully download and upload a torrent with my local KTorrent, so the next steps are to actually get peers from the tracker, and some algorithm to download pieces and store them in a file.
Hope this gives some insights. Let me know if you have specific questions about the process.