Comment by pie_flavor
6 days ago
I have one very specific retort to the 'you are still responsible' point. High school kids write lots of notes. The notes frequently never get read, but the performance is worse without them: the act of writing them embeds them into your head. I allegedly know how to use a debugger, but I haven't in years: but for a number I could count on my fingers, nearly every bug report I have gotten I know exactly down to the line of code where it comes from, because I wrote it or something next to it (or can immediately ask someone who probably did). You don't get that with AI. The codebase is always new. Everything must be investigated carefully. When stuff slips through code review, even if it is a mistake you might have made, you would remember that you made it. When humans do not do the work, humans do not accrue the experience. (This may still be a good tradeoff, I haven't run any numbers. But it's not such an obvious tradeoff as TFA implies.)
I have to completely agree with this and nobody says this enough.
This tradeoff of unfamiliarity with the codebase is a very well understood problem for decades. Maintaining a project is 99% of the time spent on a successful project.
In my opinion though, having AI write the initial code is just putting most people in a worse situation with almost no upside long term.
I agree I'm bullish on AI for coding generally, but I am curious how they'd get around this problem. Even if they can code at super human level, then you just get rarer super human bugs. Or is another AI going to debug it? Unless this loop is basically fail proof, does the human's job just becoming debugging the hardest things to debug (or at least a blindspot of the AI)
This comment reminds me of the old idiom (I cannot remember who is credited with it) that you should be careful not to use your full abilities writing code, because you have to be more clever to debug code than you were to write it.
This type of issue is part of why I've never felt the appeal of LLMs, I want to understand my code because it came from my brain and my understanding, or the same said of a teammate who I can then ask questions when I don't understand something.
1 reply →
I haven't seen enough mention of using these tools to generate formal verification specs for their output, like TLA+. Of course, you're stuck with the same problem of having to verify the specs but you'll always be playing this game and it'd seem like this would be one of best, most reassuring ways to do so.
I'll have the look into this some more but I'm very curious about what the current state of the art is. I'm guessing it's not great because so few people do this in the first place -- because it's so tedious -- and there's probably not nearly enough training data for it to be practical to generate specs for a JavaScript GQL app or whatever these things are best at generating.
> becoming debugging the hardest things to debug
This is my current role, and one of the biggest reasons AI doesn't really help me day to day agent or otherwise.
In my ideal world, AI become so proficient at writing code that they eventually develop their own formally verifiable programming language, purpose built to be verifiable. So that there wouldn't be room for unknown unknowns.
> When stuff slips through code review, even if it is a mistake you might have made, you would remember that you made it.
I don’t know. Ever had the experience of looking at 5+ year old code and thinking “what idiot wrote this crap” and then checking “git blame” and realising “oh, I’m the idiot… why the hell did I do this? struggling to remember” - given enough time, humans start to forget why they did things a certain way… and sometimes the answer is simply “I didn’t know any better at the time, I do now”
> You don't get that with AI. The codebase is always new.
It depends on how you use AI… e.g. I will often ask an AI to write me code to do X because it gets me over the “hump” of getting started… but now this code is in front of me on the screen, I think “I don’t like how this code is written, I’m going to refactor it…” and by the time I’m done it is more my code than the AI’s
Understanding code takes more effort than writing it, somehow. That's always been a huge problem in the industry, because code you wrote five years ago was written by someone else, but AI coding takes that from "all code in your org except the code you wrote in the past couple years" to "all code was written by someone else".
How well does your team work when you can't even answer a simple question about your system because nobody wrote, tested, played with the code in question?
How do you answer "Is it possible for our system to support split payments?" when not a single member of your team has even worked on the billing code?
No, code reviews do not familiarize an average dev to the level of understanding the code in question.
> Understanding code takes more effort than writing it
yes!
> somehow
not difficult to explain. Coding is a creative activity where you work top-down; you decompose the abstract/high-level into the detailed/low-level. You dictate the "events" happening to the code, you are in control. Reviewing is reactive; the code you review dictates what happens in your brain (you are under control, not in control), and you need to work bottom-up: you try to re-compose the whole from the fragments. Even for human coders, a detailed commit message is a pre-requisite before their code can undergo review. The reviewer is in the worse position, so he needs to be supported as much as possible.
Oddly, I don't tend to get that experience very much. More often, it's "That's not how I'd naively write that code, there must be some catch to it. If only I had the foresight to write a comment about it..." Alas, I'm still not very good at writing enough comments.
Now this is where AI assisted coding shines in my opinion.
I find myself both:
- writing a comment so that Copilot knows what to do
- letting Copilot write my comment when it knows what I did
I'm now a lot more reliable with my comment writing.
> why the hell did I do this? struggling to remember
- git blame
- always write good commit messages
5 years? You’re a genius. I can’t make sense of stuff I wrote last week.
Similar to almost self driving cars where you are still responsible. You're asking someone to do nothing at all other than being highly alert for long periods of time. That's just not how people work. There is no way someone can be ready to take over in an instant without actively engaging in the driving.
I read a study[1] (caveat, not peer reviewed yet I don't think?) that seems to imply that you are correct.
[1]: https://www.microsoft.com/en-us/research/wp-content/uploads/...
This is a good point I think, and these steps take time and should definitely be done. I'm not sure people take this into account when talking about having AI code for them.
So do the thing that a student copying their notes from the board does: look at the PR on one monitor, and write your own equivalent PR by typing the changes line-for-line into your IDE on the other. Pretend copy/paste doesn’t exist. Pretend it’s code you saw in a YouTube video of a PowerPoint presentation, or a BASIC listing from one of those 1980s computing magazines.
(And, if you like, do as TFA says and rephrase the code into your own house style as you’re transcribing it. It’ll be better for it, and you’ll be mentally parsing the code you’re copying at a deeper level.)
You still didn't have to build the mental model, understand the subtle tradeoffs and make the decisions that arrived at that design.
I'm amazed that people don't see this. Absolutely nobody would claim that copying a novel is the same thing as writing a novel.
I feel like the dismissal of mental models is a direct consequence on the tech industry's manaical focus on scale and efficiency as the end-all be-all values to optimize.
Nevermind other important values like resilience, adaptability, reliability, and scrutability. An AI writes a function foo() that does a thing correctly; who has the know-how that can figure out if foo() kills batteries, or under what conditions it could contribute to an ARP storm or disk thrashing, or what implicit hardware requirements it has?
Hunter S. Thompson claimed to have re-typed The Great Gatsby because he wanted to know what it feels like to write a great novel.
Apparently, he actually meant this as a somewhat serious piece of writing advice, but I still prefer my initial reading of it as sarcasm.
I am suspicious of this argument, because it would imply that you can’t understand the design intent / tradeoffs / etc of code written by your own coworkers.
Which: of course you can. Not least because both your coworkers and these coding agents produce changes with explanatory comments on any lines for which the justification or logic is non-obvious; but also because — AI PR or otherwise — the PR consists of commits, and those commits have commit messages further explaining them. And — AI submitter or otherwise — you can interrogate the PR’s submitter in the PR’s associated discussion thread, asking the submitter to justify the decisions made, explain parts you’re suspicious of, etc.
When you think about it, presuming your average FOSS project with an open contribution model, a PR from an AI agent is going to be at least strictly more “knowable” than a “drive-by” PR by an external one-time contributor who doesn’t respond to discussion-thread messages. (And sure, that’s a low bar — but it’s one that the average accepted and merged contribution in many smaller projects doesn’t even clear!)
1 reply →
This is how a (video game) programming class in my high school was taught. You had to transcribe the code from a Digipen book.... then fix any broken code. Not entirely sure if their many typos were intentional, but they very much helped learn because we had no choice but to correct their logic failures and taypos to move onto the next section. I'm still surprised 20 years later how well that system worked to teach and push us to branch our understandings.
Yes, I was just about to say. Typing out code is a way to lear syntax of a new language and it’s often recommended to not copy paste while you start learning.
Is this just repeating labor? Why not just write it all yourself in the first place if you are just going to need to copy it over later?
> The codebase is always new. Everything must be investigated carefully.
That's dreadful. Not only is familiarity with the code not valued, it is impossible to build for your own sake/sanity.
100%, i had gemini write code for a blog in golang - it has some bugs and it took me a some time to find them.
To me the sweet spot is, i write the code with the "Help" of an LLM. It means i double check everything it generates and prompt it to write code block by block - frequently acting as an editor.
Either you want human intervention for correctness and extension or you don't. Having LLM's write large swaths of code is like completely relying on tesla's autopilot - you are probably more stressed than if you just drove yourself.
The careful vetting of code and thoroughly testing it is super important, I would never even think of putting any generated code into any use without doing that.
Also your last comparison made me chuckle, good one :)
+1
Writing code is easier than long term maintenance. Any programmer is able to write so much code that he will not be able to maintain it. Unless there are good AI tools helping with maintenance there is no point to use generative tools for production code. From my experience AI tools are great for prototyping or optimizing procrastination.
This level of knowledge is nearly impossible to maintain as the codebase grows though, beyond one or two people at a typical company. And tools need to exist for the new hire as well as the long-standing employee.
Welcome to project architecting, where the job isn't about putting more lines of code into this world, but more systems in place to track them. A well layered and structured codebase can grow for a very long time before it becomes too hard to maintain. And generally, the business complexity bites before the algorithmic one, and there's no quick fix for that.
It's cultural too. I've heard people say along the lines "we don't ship the org chart here" in a positive light, then in a later meeting complain that nobody understands what's going on in their owner-less monorepo.
Shipping the org chart isn't the only way to solve this problem but it is one that can work. But if you don't acknowledge the relationship between those problems, AGI itself probably isn't going to help (partially sarcastic).
Talking to an LLM feels like talking to Leonard Shelby from memento. https://youtube.com/watch?v=Y3kNTvXVHvQ
AI tools are enabling the developer to a higher level of abstraction to engineering manager or product manager. Those roles do not need to be familiar with code in that detail.
This is it. Reading AI slop does not form synapses in your brain like writing the code yourself does.
Exactly. See also https://hazelweakly.me/blog/stop-building-ai-tools-backwards... for a detailed look at this aspect of AI coding.
[flagged]
It might do it later... but odds are it'll requiring ingesting a post a human made about the exact problem while listing out what the correct output should have been.
The problem is - What happens when you ask a question that you have no idea what the correct answer might be? Sounds like a really big Wack-a-Mole to me.
I do find it useful - maybe not quite as useful as the people that stand to make gobs of money off it think it is but useful non the less.
Companies that go out and dump large numbers of employees and replace them with AI are likely in for a rude awakening (if those companies actually exist). If they can actually do that the company was not really offering much value to begin with.
The important thing you are missing is that the learning landscape has now changed.
You are now responsible for learning how to use LLMs well. If an untrained vibe coder is more productive for me, while knowing nothing about how the code actually works, I will hire the vibe coder instead of you.
Learning is important, but it's most important that you learn how to use the best tools available so you can be productive. LLMs are not going away and they will only get better, so today that means you are responsible for learning how to use them, and that is already more important for most many roles than learning how to code yourself.
This is actually a good reason for exiting the industry before one's job goes away. Steering AI to barf up the right-looking pool of vomit is not the Flow-generating experience that many people have started to program for.
There is room to move up the developer hierarchy at the company I work for, but I refuse to take that path for this very reason. The leadership has bought into AI as some kind of panacea, plus the company's plans to replace hundreds of human administrators in our B2C operations with AI strike me as downright evil.