← Back to context

Comment by seanmcdirmid

16 hours ago

I’m currently on the beach at Google (since mid July), so I’m looking internally and externally for a new role. But I’m exhausted, so I’m playing around with local LLMs and DeepSeek more as well. I’m currently trying to get an agent harness going where orchestration happens via bazel, and agents are completely isolated beyond what context you give them access to as expressed in starlark. Agent executions are organized as a DAG like a build would be, where the output of one agent can serve as context for another (there is also a feedback mechanism which is needed for arbiter nodes).

I’ve mentioned it before, but we can get some pretty good results for developing implementation and tests with this setup. The context isolation makes it easy to separate test and code writing using a specification as a source of truth. Tests are then run by an arbiter that feedback to tests or implementation depending on who it blames for test failures. The tests are then treated as an independent implementation, and through the math of coincident errors, the arbiter feedback loop ensures that both are eventually correct (N-versions and clean room research did this with humans in the 80s). This then allows us to use less reliable LLMs to develop code, like Qwen on a MacBook Pro.

I hope to have something out on GitHub before my beach period ends (well, if I’m unsuccessful in getting a new role, then I’ll have a lot more time to play at least).

Lot of people are building agent runtime platforms right now. Look at LiteLLM's play into the space, for example. It's the new hot space for innovation. IMO the main play here is going to be on the self hosted/provenance aspect, because every major provider is now branching into services now to try and entrap you into their hosted platform ecosystem with all of these integrations

  • I actually considered lite LLM to get Qwen to work with codex. It mostly works without lite LLM but MCP servers aren’t available, lite LLM tricks into think Qwen is an OpenAI model.

Does it need to be actual Bazel or could it be a similar but simpler, purpose-built system?

  • No, it could be python or a DSL or whatever. Using Bazel and starlark is just a shortcut given a lack of any good ideas yet for a purpose built orchestration language.

Does on the beach mean gardening leave? I'm not familiar with the term.

  • Ya, it means my role at Google is gone and I have to find a new role or be laid off. I think “on the beach” sounds better than gardening leave, the term comes from the American consulting firms.

    • That’s weird. Isn’t that kind of on the employer to decide whether they have different work for you? Are you expected to be actively doing something during this period?

      1 reply →