Comment by luckystarr
15 hours ago
The way I use AI now feels more exhausting than the programming I did for the last 20 years. I pose a problem, then evaluate proposals, then pick the one I think is the "right one"(tm), then see the AI propose a bunch of weird shit, then call it out, refine the proposal until it feels just about right (this is the exhausting part), then let it code the proposal. The coding will then run for 1-5 hours and produce something that would have taken me at least 2 or 3 weeks (in that quality).
After 5 hours or so of doing this planning, I'm EXHAUSTED. I never was exhausted in this manner from programming alone. Am I learning something new? Feels like management. :)
I feel this as well. I think it’s something to do with having to be more “on” as you slowly work with the LLM to define the problem and find a reasonable solution. There’s not much of a flow-state. You have to process mountains of output and identify the critical points, over and over, endlessly. And it will always be an off in this unsettling little way, even when it’s mostly quite good. It’s jarring.
The strange sorts of errors and reasoning issues LLMs have also require a vigilance that is very draining to maintain. Likewise with parsing the inhuman communication styles of these things…
Could it be that what we called flow state was actually a sort of high level thinking time afforded by doing low level routine work?
For instance I'm the old world, if you wanted to change an interface, you might have to edit 5 or 6 files to add your new function in the implementations. This is pretty routine and you won't need to concentrate that much if you're used to it, so you can spend that low-effort time thinking about the bigger picture.
you may be right on this hunch. but I think the old world is no longer there now :( more thinking is expected per unit time
Its the "unsettling little ways", right. So you can't skip whole paragraphs, you literally have to read everything. And sometimes its worded in ways I don't understand at all (due to missing implications that the LLM conveniently omitted), so I have to re-ask it about that point as well. For every major feature or work-unit it takes up to 2 or 3 hours.
I figured out some patterns in the way it behaves and could put more guard-rails in place so they hopefully won't bite me in the future (spelled out decision trees with specific triggers, standing orders, etc.), but some I can't categorize right now.
How do you check if what it produced is even the right thing? Models love to go chasing the wrong goal based on a reasonable spec.
When the end result has problems and needs to be reworked.
You can't figure this out instantly except when you'd review everything the LLM produces, which I am not. So the round trip time is pretty long, but I can trace it back to the intent now because I commit every architecture decision in an ADRs, which I pour most of my energy into. These are part of the repo.
Using these ADRs helped a lot because most of the assumptions of the LLM get surfaced early on, and you restrict the implementation leeway.
Got it. I imagine concurrency bugs will hit hard with this approach because they show up rarely and are hard to debug.
Do they? I haven't experienced models deviating from a spec in a very long time. If anything I feel they are being too conservative and have started to ask to confirm too much.
The problem is not the LLM deviating from the plan (though that rarely also happens when it thinks it has a better idea) but rather if the plan is not strict enough and the LLM decides on the fly HOW it is going to build your plan.
Sounds like you’re using Waterfall Which, if it works for you, go for it. But maybe Agile would feel more dynamic.
To me it’s more like being a super micro-managing TL that would annoy the hell out of their human reports. It comes with all the pros and cons of micro-management.
I think one of the benefits of AI is that it will get started, and keep going.
But maybe pacing/procrastination might be relief valves?
AI does the easy/medium part, leaving only hard stuff and context switching, so naturally it's more exhausting, as the concentration of difficult-work-per-unit-time and context-switching-per-unit-time is much higher.
[dead]