Comment by zeendo

Note that this is a team method, not an individual one. For my personal (non-commercial) development work, I still use LLM one-on-one - but for commercial team work, this common terminal method has been a productive method, also.

We don’t use coding agents - these are incompatible with security and IT methods we already use very successfully to maintain stringent quality demands (our product target is heavy industrial/life-critical services) - we mostly interact with the LLM on existing codebases, and strictly in sandboxed scenarios through a robust and proven git workflow.

There is, literally, a single physical terminal which any and all of us use - think of the LLM like a refrigerator, not a footlocker. This machine is isolated from everything else in our lab, except the repo in focus. It is the only LLM interface for our projects, and it is a common lab resource - developers use it when they need to, communicating back to their main workstations through the git workflow.

Yes, there is only one context window, everyone looks through it. This reduces and massively simplifies the tooling load.

Tooling and models: Firefox for web-based queries, git and hermes for everything else, tmux for the front-end, and it is of course a virtual machine, anyone can remote to, share, observe, pair-program - and there are a set of models in use, for different types of problems encountered by the team members.

Everyone pair programs anyway already, and code reviews are treated as important as standups also, LLM or otherwise. No LLM code can be committed to upstream branches without at least 2 reviews within the team.

The team is about 12 members, mostly senior developers with decades of experience delivering safety-critical, realtime systems, give or take the odd intern or student being added/elsewise assigned over the past year.

It took, literally, a week before we sorted this out.

We all had LLM rigs in our home labs (and of course we all still do), but when it comes to maintaining the collective corpus, so to speak, we realized we would get more out of the LLM if we all spoke to the same endpoint.

I think it was important that our git flow was already circumspect; certainly we have been able to treat the LLM as we treat our interns - everyone has access to the common fridge/kitchen, so the intern/LLM learn how to cook faster/how to do things properly.

There are some other principles, which I will summarize, at play:

1. All AI/ML is to be contained - it is not just widespread installed in the environment. Humans are the best container for AI/ML. We (senior developers corps, anyway) all had the individual conclusion that this must be The Way™, as we all got bit by the AI/ML bug. The best way for humans to contain the LLM, is to discuss the LLM. Having the same common interface to the LLM, makes it a lot easier for humans to discuss what is going on with the LLM.

2. The AI/ML must be controlled precisely in order to apply it to the problem, properly. This takes real human experience. It is easier to do if the team shares the experience. Thus, we have the common historiography reviews, which we treat as just as important as team standups where other things are discussed. We discourage prompt copying - everyone still has their style - but instead we encourage standard terms and definitions; often-times, the history in the common LLM terminal is where a lot of us learn the real meanings for things.

3. Humans are still in the loop. Nobody is allowed to just say “the LLM wrote this code”. What is important to the team is, who reviewed it? Who reviewed it, again?

That has saved our ass so many times it’s just muscle memory now.

4. The best LLM is a local LLM. We now feel confident, as a team, to take on the task of training our own model and using it for broader development in future products. The process in place to do it properly is simply the general consensus being maintained by the common team reference.