Comment by repelsteeltje
4 days ago
> I’m interested in connecting with senior engineers who enjoy building AI systems in Rust, think in terms of orchestration and state, and care deeply about correctness, performance, and privacy.
So something like triton server backends, but using rust rather that python or c++?
Thanks. That’s a fair comparison, but I’m thinking a bit broader than model serving alone.
I’m less interested in building a generic inference backend and more focused on how the system behaves over time — things like agent lifecycles, state, memory, and coordination. Inference is part of it, but not the whole thing.
Correctness, performance, and privacy matter because the system is long-lived and stateful, not just because it’s pushing tokens around.
Sounds ambitious and exciting. How do you intend to provide support to users given that the system will likely evolve and diverge beyond what you can reasonably test for during development? Will you be able to ship software updates without introducing regressions?
Good question — this is one of the tricky parts.
I don’t think you can support something like this if it’s a total black box. If behavior drifts, users need some way to see what state the system thinks it’s in and why it did what it did, otherwise support becomes guesswork.
On updates, I’m pretty cautious about changing behavior silently. I’d expect versioned behavior, explicit migrations, and being careful about what auto-updates versus what users opt into.
And if people are paying over time, updates need to be real improvements they can understand, not just maintenance or regressions.
This feels more like a design problem than a testing problem.
1 reply →