Comment by manbart
1 day ago
How is the software compatibilty with the Tenstorrent cards? Are you stuck using vendor supplied runtimes/models?
It's surprising how little these things come up given the price they go for
1 day ago
How is the software compatibilty with the Tenstorrent cards? Are you stuck using vendor supplied runtimes/models?
It's surprising how little these things come up given the price they go for
The software stack is pretty immature, definitely very DIY. Their officially supported models are pretty old at this point, though there’s community support for gemma4, and models with GDN like qwen3.6 is supposedly very close.
The entire stack (minus some binary blobs in firmware) is open source, so if you have the time and persistence you can get whatever you want done.
A few community members have been working on support with llamacpp, where we can have supported operations offloaded to the TT cards, while having unsupported ops running on GPU or CPU. Llamacpp is pretty good at that. The existing kernels could definitely be better, and I’ll try my hand at writing some kernels some time.