Comment by trwhite
8 hours ago
I'm really struggling to see how to make architectural decisions with an agent. It's great when you're at a total loss for ideas, but when you already have some of the pieces it ultimately wants to drive all of the thinking and takes over. Then it just feels like you're deferring your experienced judgement. I've seen colleagues lose the ability to reason any more without asking the agent to do it for them, because they inherently don't see the point if the agent is going to end up doing the whole piece (and probably auditing/overruling anything they came up with on their own).
I tend to find this part of the work enjoyable. It is just a faster and more productive version of what I would do with any engineer working for me who owns a large feature. I barely have to hint at my concern or drop the right keyword and the agent (Fable/Astra) will immediately understand.
"Couldn't this be stateless?" "Do you have a plan to be able to shard this?" - We will almost always pivot from the agents initial design but the agent is able to easily understand the reasons and benefits and align quickly.
If I was writing the code myself, I'd often have to make compromises between the ideal architecture and the level of effort required to implement it - now I can just always have the ideal architecture.
> I can just always have the ideal architecture.
Now you have what you think is the ideal architecture. Since you didn't implement it, you didn't discover it was not the ideal one mid way into the implementation.
May be it is too complicated, but you wouldn't know, because LLM is doing the implementation. If you did implement it yourself, you might have spotted a critical point that might simplify the whole thing...
If you did implement it yourself, you might have spotted a critical point that might simplify the whole thing...
This is the "Jodie Foster in Contact listening for the SETI signal with headphones" theory of how production software systems work.
4 replies →
This has been my experience as well. LLMs know the patterns but often need to be nudged into choosing the proper one.
> I'm really struggling to see how to make architectural decisions with an agent. It's great when you're at a total loss for ideas
I feel the opposite, like if I go to an agent without first knowing what I want to build, I'll never figure out what I'm doing or why and it'll run away from me.
I pretty much always go back and forth and have the agent write out a plan to a file and review it myself in my text editor. I still sometimes end up with surprises I disagree with, but I don't really find it to be true that the LLM ends up trying to "drive all of the thinking".
When I'm thinking through an architecture, I not only instruct it to refrain from writing any code, I don't even necessarily tell the agent what I'm trying to build.
> I'm really struggling to see how to make architectural decisions with an agent.
I walk to work and home with ChatGPT Voice and AirPods. I ask it to be Socratic and I just start rambling the top of thing on my mind. After 20 mins of back-and-forth it's usually teased an answer out of me or I've teased an answer out of it.
What kind of discussions/thoughts is this about?
And do you feel comfortable with the setting that a corporation has a detailed log of your deepest thoughts?
This is one thing I actually do with a chatbot, instead of an agent. Voice mode even.
I start talking to it while doing menial tasks like cleaning or doing the laundry, and I discuss the architectual decisions and options until I come to some resemblance of a plan.
Good side of this approach is that I can't just "skim over" or "copy paste" things - either I understood them and can repeat them myself, or I can't. It takes more time than /grill-me and similar approaches, but it's the only approach that doesn't make me want to claw my brain out.
I do the same, but without AI in the loop. Just me and my thoughts while cleaning or doing laundry or going for a run. The best designs and architectural choices and algorithms I've come up with in my carreer were developed and/or refined that way.
It's very enlightening, though it requires being comfortable with being alone with ones thoughts. It seems to me many people are not and need constant distractions and/or dopamine kicks.
+1. I have found that most coding harnesses are too focused on the doing that using a Claude/chatgpt chat gives me a lot better quality.
Sometimes it's easier for me to scaffold the architecture (in real, working code) and then let the agent fill in the implementation. And I make the agent document the architecture and have it refer to the documentation when coding. When I review agent code, I focus only on architecture (is it following existing architecture? is it introducing new structures, dependencies, etc.?)
Other times, I let the agent create a black box with a well-defined interface contract. I don't care about the architecture inside the black box.
How much scope do you give it? I find it's easier to stay in control when you give it scope down the chunks of work you give it.
Why would you care about architecture? It was an issue when humans were writing code so architecture mattered in the sense that you needed a sweet spot between current requirements and future extensions.
The agent can rewrite half your codebase in one day, so architecture stops mattering for the most part.
Because even if the rewrite worked, and worked perfectly, you've (a) lost a day, and (b) there's no guarantee that you won't have to do that all over again in a few days' time, because you're still not thinking about the architecture.
But if the codebase is large enough, and if you don't have tests for everything, you'll likely lose stuff along the way. So you'll then spend at least another day tidying up the rewrite, just getting it back to where you were two days ago.
Does that sound like fun? Wouldn't it be easier to just plan it correctly the first time?
Almost all scalability and performance optimization is architectural in nature. AI writing the code doesn't eliminate this concern.
Some apps don't care about scalability and performance but many do. Ignoring architecture all but guarantees inefficient, wasteful software.
Why try to make things that are good? Why care about anything? Why not just turn our minds over to the machines, and let them rock us to sleep like babies?
Possibly because the goal is the end product that serves a purpose and has value, not the process. If you can make your product, make the customer happy, and make your money to enjoy your life and secure your families future… why care who or what or how (as long as it’s ethical obviously).
4 replies →
An agent can rewrite a small codebase in half a day, I have millions of lines of code - it can't rewrite that in half a day.
Abstractions. They make it easier for a human to understand. We need abstractions to fit systems into our working memory. Agents dont.
Agents don’t have infinite working memory…
LLMs benefit from abstractions for the same reasons that humans do. More information in the same amount of text. Fewer working parts to juggle so fewer ways to make mistakes.
How are you going to read the code if it's not structured in a way you understand?
I think the argument is that there won't be a reason to read the code. If you want a change, just let the agent change it.
Similar to the output of a compiler. Nobody (with very few exceptions) reviews its machine code output. No reason to do that. If you want to change it, just recompile.
(I don't share this view, but I think a substantial and growing fraction of folks does.)
Mhh I've tried one-shotting double buffered IO a few times and it just can't do it. Maybe it's a prompting issue, maybe LLM's have a hard time with thinking about parallel processes.
I'll create a simple "framework" of what I know works. After that's there the LLM is fantastic.
Why would you care about architecture? Ability to implement the current feature, ability to implement future features, maintainability, and (at least sometimes) performance.
But why would you care when an AI can just rewrite it? Yes, but can it rewrite it to a good architecture? Or just to a different one?
Does a good architecture make code easier for an AI to maintain? I don't know, but I think it's at least not proven that it doesn't.
It’s important for separation of concerns, which is important for maintenance and future iteration on features and bug fixes. If you don’t have separation of concerns, every change to anything is essentially a from scratch rewrite of the whole thing. That’s also incredibly inefficient in token usage.
I have had really good experiences with designing architecture with agents. It is much much better than humans and frankly, if my colleagues had used agents to design their new services, we'd have been in much better place.
I have found autonomous agents to be pathetic at architectural decisions.
Architecture has remained so far to be one of those domains where skillset dwarfs everything by comparison.
Treat the final "here's what you should do" summary as just another intrusive thought.