Comment by indiv0
5 hours ago
Funny. I started working on a similar product myself just a few days ago. I have multiple machines for multiple different purposes. Each one runs different agents (some local, some cloud), for different tasks. Different ones store different data (a NAS on one, my VMs on another, etc.). I wanted a durable layer over all of them to be able to start work on any one of them and resume on any other, and for all the disparate agents to share memory/configurations where possible, and for each node in the network to make all the others aware of its purpose (so when the NAS is online, the agents know how to connect to it and what it contains). Sadly no VC funding for me, building it on my lonesome :)
Sounds super interesting!
What's your approach?
Still exploring different options but at the moment I run a daemon on each node that connects via iroh to link the nodes (and NAT traversal). Once the link is established the nodes share data via a Raft-based datastore. I can launch a TUI client on any machine. This presents a chat interface like a typical LLM harness. From here I can configure which model I want to use, how the model is executed and on which node (for example I can use Ollama on machine A to serve a model that’s running on machine B, but executing commands via this system on machine C). Each daemon then acts like a typical LLM harness, but headless. So the commands and tool calls are streamed into the shared store and finally end up in my TUI. Since all the state is in the distributed store I can pick up the progress wherever I want, as long as there’s a node running there.
Still a lot of big problems like latency so I’m not too happy with the current design. It’s a LOT of state to replicate, and obviously things get problematic when an agent was working on a project on machine A but then ends up on machine B which doesn’t have that project directory around at all. So the agent needs to re-orient itself to figure out what happened. Still pretty fun to play with.