Comment by ericwood
1 year ago
I have a hard time being sold on “yea it’s wrong a lot, also you have to spend more time than you already do on code review.”
Getting to sit down and write the code is the most enjoyable part of the job, why would I deprive myself of that? By the time the problem has been defined well enough to explain it to an LLM sitting down and writing the code is typically very simple.
You're giving the game away when you talk about the joy LLMs are robbing from you. I think we all intuit why people don't like the idea of big parts of their jobs being automated away! But that's not an argument on the merits. Our entire field is premised on automating people's jobs away, so it's always a little rich to hear programmers kvetching about it being done to them.
I naively bought into the idea of a future where the computers do the stuff we’re bad at and we get to focus on the cool human stuff we enjoy. If these LLMs were truly incredible at doing my job I’d pack it up and find something else to do, but for now I’m wholly unimpressed, despite what management seems to see in it.
Well, I've spent my entire career writing software, starting in C in the 1990s, and what I'm seeing on my dev laptop is basically science fiction as far as I'm concerned.
6 replies →
We get to do cool stuff still, by instructing the LLM how to build such cool stuff.
> Our entire field is premised on automating people's jobs away, so it's always a little rich to hear programmers kvetching about it being done to them.
Watch out, you’re giving your game away.
My job is about enabling analysis that was previously done ad hoc and informally. If I’m harming people then that’s something I have to take responsibility for, but it’s also caused not by my direct contribution but by the larger system that I’m working within.
I expressively don’t want to automate away work when that will just result in more profit for private owners and less income for regular working people.[1] And I also don’t want to automate work if that means shifting drudgery to some worker to fill in that freed up time.
And how does this contradict what “we” are doing and stand for!? We criticize technology on this board all the time!
But it’s nice to have the priorities of such a prominent member on the record.
[1] But I DO want to automate work in the hypothetical society where we all own the automation and thus the only thing we are deprived of is drudgery.
The parts worth thinking about you still think about. The parts that you’ve done a million times before you delegate so you can spend better and greater effort on the parts worth thinking about.
This is where the disconnect is for me; mundane code can sometimes be nefarious, and I find the mental space I'm in when writing it is very different than reviewing, especially if my mind is elsewhere. The best analogy I can use is a self-driving car, where there's a chance at any point it could make an unpredictable and potentially fatal move. You as the driver cannot trust it but are not actively engaged in the act of driving and have a much higher likelihood of being complacent.
Code review is difficult to get right, especially if the goal is judging correctness. Maybe this is a personal failing, but I find being actively engaged to be a critical part of the process; the more time I spend with the code I'm maintaining (and usually on call for!) the better understanding I have. Tedium can sometimes be a great signal for an abstraction!
The parts I've done a million times before take up... maybe 5% of my day? Even if an LLM replaced 100% of this work my productivity is increased by the same amount as taking a slightly shorter lunch.
> The parts that you’ve done a million times before you delegate
That's where I'm confused. I've been coding for more than 20 years, and every task I ever did was different from the other ones. What kind of task do you do a million times before realizing that you should script it in bash or Python?
I'm confused when people say that LLMs take away the fun or creativity of programming. LLMs are only really good at the tedious parts.
First of all, it’s not tedious for a lot of us. Writing characters themselves is not a lot of time. Secondly, we don’t work in a waterfall model, even on the lowest levels, so the code quantity in an iteration is almost always abysmal or small. Many-many times it’s less than articulate it in English. Thirdly, if you need a wireframe for your code, or a first draft version, you can almost always copy-paste or generate them.
I can imagine that LLM is really helpful in some cases for some people. But so far, I couldn’t find a single example when I and simple copy-pasting wouldn’t have been faster. Not even when I tried it, not when others showed me how to use it.
Because the tedious parts was done long ago while learning the tech. For any platform/library/framework you've been using for a while, you have some old projects laying around that you can extract the scaffolding from. And for new $THING you're learning, you have to take the slow approach anyway to get its semantic.
For me it's typically wrong not in a fundamental way but a trivial way like bad import paths or function calls, like if I forgot to give it relevant context.
And yet the time it takes me to use the LLM and correct its output is usually faster than not using it at all.
Over time I've developed a good sense for what tasks it succeeds at (or is only trivially wrong) and what tasks it's just not up for.