Thanks for the encouragement, if you have the hardware and end up trying it out I'd love to get your feedback (feel free to drop an issue in the repo) :)
I would love to do that, but unfortunately don’t have one on hand (yet?). In fact, I’m curious - did you buy a TT device to work on this, or are you using a cloud provider like Koyeb?
I first built my own to try it out (the exact hardware is described in https://relativity.cc/hardware/ under "one Blackhole card") and when I liked the hardware I got a quietbox 2. It is very high quality but a little expensive (but inexpensive compared to other LLM hardware and it has excellent interconnects which is very rare in the consumer/dev space).
Thanks for the encouragement, if you have the hardware and end up trying it out I'd love to get your feedback (feel free to drop an issue in the repo) :)
I would love to do that, but unfortunately don’t have one on hand (yet?). In fact, I’m curious - did you buy a TT device to work on this, or are you using a cloud provider like Koyeb?
I first built my own to try it out (the exact hardware is described in https://relativity.cc/hardware/ under "one Blackhole card") and when I liked the hardware I got a quietbox 2. It is very high quality but a little expensive (but inexpensive compared to other LLM hardware and it has excellent interconnects which is very rare in the consumer/dev space).