What if I just enjoy how I work at the moment and don't really care about this stuff? Why do I _have_ to give it a go? Why don't LLM evangelists accept this as an option?
Choosing not to use AI agents is maybe the only tool position I feel I've had to defend or justify in over a decade of doing this, and it's so bizarre to me. It almost reeks of insecurity from the Agent Evangelists and I wonder if all the "fear" and "uncertainty" they talk about is just projecting.
Nobody pushed you to use git when you were comfortable with svn? Nobody pushed you to use Docker when you were comfortable running bare metal? Nobody pushed you to write unit tests when you were comfortable not? Nobody pushed you to use CSS for layout when you were happy using tables?
Some of those are before your time, but: The only time you don't get pushed to use new technologies is when a) nothing is changing and the industry is stagnant, or b) you're ahead of the curve and already exploring the new technology on your own.
Otherwise, everyone always gets pushed into using the new thing when it's good.
The engineers using svn were the ones who were pushing for git - I was the one saying "we can't, because none of the conversion tools competently preserve branch history, and it's even worse on repos that started in CVS". Noone responsible for repos was pushing for git, it was end-users pulling for it (and shutting up when they learned how much work it would cause :-) That looked nothing like the drug-dealer-esque LLM push I've been seeing for the last 3 years.
(Likewise with CVS to svn: "you can rename files now? and branches aren't horrible? Great, how fast can we switch?" - no "pushing" because literally everyone could see how much better it was in very concrete cases, it was mostly just a matter of resource allocation.)
In the context of this discussion, it feels more like ipv6 :-)
My own experiences with LLMs have shown them to be entertaining, and often entertainingly wrong. I haven't been working on a project I've felt comfortable handing over to Microsoft for them to train Copilot on, and the testimonials I've seen from people who've used it are mixed enough that I don't feel like it's worth the drawbacks to take that risk.
And...sure, some people have to be pushed into using new things. Some people still like using vim to write C code, and that's fine for them. But I never saw this level of resistance to git, Docker, unit tests, or CSS.
I'm the same as you: I don't really care about this stuff. Given what we know about the start of these AI endeavors (torrenting/scraping the whole internet and training on it) then running their software seems like a liability. Is it pasting code verbatim that falls under an incompatible license? Is it training on my codebase? Why would I want to depend on this very compute intensive, cloud hosted tool?
At least with other advancements in our field like git, Docker, etc., they're made with a local-first mindset (e.g. your git repos can live anywhere and same with your docker images)
You don’t have to, of course, but you probably will if you want to be competitive in a professional capacity in the future.
Not doing so seems a bit like a farmer ploughing fields and harvesting crops by hand while seeking to remain competitive with modern machinery, surely?
I've seen a lot of developer tooling change and evolve over the course of my career, but with AI it was the first time I've seen people in non-technical managerial positions trying to force the engineers to make a switch. It was extremely bizarre.
I think the standing assumption is that most of us take pride and enjoyment in being good at our craft - and some of us even want to be great at it. That means understanding all the tools at our disposal - to see if they are useful or not.
If that is not interesting to you I think that’s a totally fine choice, but you’re getting a lot of pushback from people who have made a different choice.
It almost reeks of insecurity from the Agent Evangelists and I wonder if all the "fear" and "uncertainty" they talk about is just projecting.
That's probably true on some level for some evangelists, but it's probably just as true that some people who are a bit scared of AI read every positive post about it as some sort of propaganda trying to change their mind.
Sometimes it's fine to just let people talk about things they like. You don't know what camp someone is in so it's good to read their post as charitably as possible.
Your work will eventually be driven by the same economics as the industry as a whole, project estimates 12 months from now will be done based on how long it takes a dev with full LLM backing, not your current speed. Then you need to be prepared to work at that speed.
> “What about security?” [..] “What about performance?” [..] “What about accessibility?”
TBH i'm fine with AI but my main concern isn't any of these issues (even if they suck now -though supposedly Claude Code doesn't- they can get better in the future).
My main concern, by far, is control and availability. I do not mind using some AI, but i do mind using AI that runs on someone else's computer and isn't under my control - and i can, or have a chance at, understanding/tweaking/fixing (so all my AI use is done via inference engines that are written in C++ that i compiled myself and are running on my PC).
Of course the same logic applies to anything where that makes sense (i.e. all my software runs locally, the only things i use online/cloud versions for are things which are inherently about networking - e.g. chat, forums, etc, but even then i use -say- a desktop-based email client instead of webmail).
Absolutely. I'm gonna go full agentic coding the day I can do it with open-weight models on my machine. Until then feeding someone else's models with more data on how to replace me in particular sounds insane to me.
where are the websites that are lightning fast, where speed and features and ads have been magically optimized by ai, and things feel fast like 2001 google.com fast
We are still massively lacking in software. We're not at the stage of making websites faster, we're at the stage of making more of them. We haven't come close to hitting the point where we have enough software and now the job is to refine it.
> I see a lot of my fellow developers burying their heads in the sand, refusing to acknowledge the truth in front of their eyes, and it breaks my heart because a lot of us are scared, confused, or uncertain, and not enough of us are talking honestly about it.
Imagine if we had to suffer these posts, day in and day out, when React or Kubernetes or any other piece of technology got released. This kind of proselyting that is the very reason there is tribalism with AI.
I don't want to use it, just like I don't want to use many technologies that got released, while I have adopted others. Can we please move on, or do we have to suffer this kind of moaning until everybody has converted to the new religion?
Never in my 20 years in this career have I seen such maniacal obsession as it has been over the past few years, the never-ending hype that have transformed this forum into a place I do not recognise, into a career I don't recognise, where people you used to respect [1] have gone into a psychosis and dream of ferrets, and if you dare being skeptical about any of it, you are bombarded with "I used to dislike AI, now I have seen the light and if you haven't I'm sorry for you. Please reconsider." stories like this one.
Jesus, live and let live. Stop trying to make AI a religion. It's posts like this one that create the sort of tribalism they rail against, into a battle between the "enlightened few" versus the silly Luddites.
The author of that post Nolan is a pretty interesting guy and deep in the web tech stack. He’s really one of the last people I’d call "tribal", especially since you mention React. This guy hand-writes his web components and files bug reports to browsers and writes his own memory leak detection lib and so on.
If such a guy is slowly dipping his toes into AI and comes to the conclusion he just posted, you should take a step back and consider your position.
I really don't care what authority he's arguing from. The "just try it" pitch here is fundamentally a tribalist argument: tribes don't want another tribe to exist that's viewed as threatening to them.
I don't think you understand how much things are about to change in a relatively short time. A lot of people are rightfully confused and concerned.
Many people are seeing this as an existential moment requiring careful navigation and planning, not just another language or browser or text editor war.
> The models don’t have to get better, the costs don’t have to come down (heck, they could even double and it’d still be worth it)
What worries me about this is that it might end up putting up a barrier for those that can't afford it. What do things look like if models cost $1000 or more a month and genuinely provide 3x productivity improvements?
If they're paying you, they can afford it. Also, even if running large teams of coding agents becomes practical, you don't necessarily need more than one or two to learn.
Today we have open source projects that can compete with proprietary ones just because people without initial funding had the ability to make it competitive
You can bootstrap something with yourself and a friend with some hard work and intelligence
This is available to people all over the world, even those in countries where $1000 is a months salary
Microsoft and their employees will be fine, yeah. That's not who I'm thinking about
Given that models seem to be converging to similar capabilities and that there are plenty of open weights models out there market competition should drive prices towards the marginal cost of inference.
I mean, your employer will pay it. $1K/month is cheap for your employer.
But there is an interesting point about what it does to hobby dev. If it takes real money just to screw around for fun on your own, it's kinda like going back to the old days when you needed to have an account on a big university system to do anything with Unix.
Are more what I had in mind. Of course an established company can pay it. I don't like the idea of a world where all software is backed by big companies
AI has pushed me to arrive at an epiphany: new technology is good if it helps me spend more time doing things that I enjoy doing; it's bad if it doesn't; it's worse if I end up spending more time doing things that I don't enjoy.
AI has increased the sheer volume of code we are producing per hour (and probably also the amount of energy spent per unit of code). But, it hasn't spared me or anyone I know the cost of testing, reviewing or refining that code.
Speaking for myself, writing code was always the most fun part of the job. I get a dopamine hit when CI is green, sure, but my heart sinks a bit every time I'm assigned to review a 5K+ loc mountain of AI slop (and it has been happening a lot lately).
I've been using it to do big refactors are large changes that I would simply avoid because, before, the benefits don't outweigh the costs of the doing it. I think half the problem people have is just using AI for the wrong stuff.
I don't see why it doesn't help with reviewing, testing, or refining code either. One of the advantages I find is that an LLM "thinks" differently from me so it'll find issues that I don't notice or maybe even know about. I've certainly had it develop entire test harnesses to ensure pre/post refactoring results are the same.
That said, I have "held it wrong" and had it done the fun stuff instead and that felt bad. So I just changed how I used it.
I agree. I’m using copilot more and more as it gets better and better, but it is getting better at the fun stuff and leaves me to do the less fun stuff. I’m in a role where I need to review code across multiple teams, and as their output is increasing, so is my review load. The biggest issue is that the people who lean on copilot the most are the least skilled at writing/reviewing code in the first place, so not only do I have more to review, it’s worse(1).
My medium term concern is that the tasks where we want a human in the loop (esp review) are predicated on skills that come from actually writing code. If LLMs stagnate, in a generation we’re not going to have anyone who grew up writing code.
1: not that LLMs write objectively bad code, but it doesn’t follow our standards and patterns. Like, we have an internal library of common UI components and CSS, but the LLM will pump out custom stuff.
There is some stuff that we can pick up with analysers and fail the build, but a lot of things just come down to taste and corporate knowledge.
I saw a similar inflection point to this guy personally, in 2024 the models weren’t good enough for me to use them for coding much, but around the time of o1/o3/Gemini 2.5 was when things changed and I haven’t looked back since.
The whole "it's turned political so it's bad" brush off that this article anchors itself on is crazy. I understand many Americans can't understand what it's like to be under threat, but I'm not pumping money into massive organizations that pay federal American taxes. And seriously, f*ck you for insinuating I should.
It is, in fact, not crazy, because none of this is predicated on using a specific vendor.
Many of these techniques can also work with Chinese LLMs like Qwen served by your inference provider of choice. It's about the harness that they work in, gated by a certain quality bar of LLM.
Taking a discussion about harnesses and stochastic token generators and forcing it into a discussion of American imperialism is making a topic political that is not inherently political, and is exactly the sort of aggressive, cussing tribalistic attitude the article is about.
I don't think everything is for certain though. I think it's 50/50 on whether Anthropic/whoever figures out how to turn them into more than a boilerplate generator.
The imprecision of LLMs is real, and a serious problem. And I think a lot of the engineering improvements (little s-curve gains or whatever) have caused more and more of these. Every step or improvement has some randomness/lossiness attached to it.
Context too small?:
- No worries, we'll compact (information loss)
- No problem, we'll fire off a bunch of agents each with their own little context window and small task to combat this. (You're trusting the coordinator to do this perfectly, and cutting the sub-agent off from the whole picture)
All of this is causing bugs/issues?:
- No worries, we'll have a review agent scan over the changes (They have the same issues though, not the full context, etc.)
Right now I think it's a fair opinion to say LLMs are poison and I don't want them to touch my codebase because they produce more output I can handle, and the mistakes they make are too subtle that I can't reliably catch them.
It's also fair to say that you don't care, and your work allows enough bugs/imprecision that you accept the risks. I do think there's a bit of an experience divide here, where people more experienced have been down the path of a codebase degrading until it's just too much to salvage – so I think that's part of why you see so much pushback. Others have worked in different environments, or projects of smaller scales where they haven't been bit by that before. But it's very easy to get to that place with SOTA LLMs today.
There's also the whole cost component to this. I think I disagree with the author about the value provided today. If costs were 5x what they are now, I think it would be a hard decision for me to decide if they are worth it. For prototypes, yes. But for serious work, where I need things to work right and be reasonably bug free, I don't know if the value works out.
I think everyone is right that we don't have the right architecture, and we're trying to fix layers of slop/imprecision by slapping on more layers of slop. Some of these issues/limitations seem fundamental and I don't know if little gains are going to change things much, but I'm really not sure and don't think I trust anyone working on the problem enough to tell me what the answer is. I guess we'll see in the next 6-12 months.
> I do think there's a bit of an experience divide here, where people more experienced have been down the path of a codebase degrading until it's just too much to salvage – so I think that's part of why you see so much pushback.
When I look back over my career to date there are so many examples of nightmare degraded codebases that I would love to have hit with a bunch of coding agents.
I remember the pain of upgrading a poorly-tested codebase from Python 2 to Python 3 - months of work that only happened because one brave engineer pulled a skunkworks project on it.
One of my favorite things about working with coding agents is that my tolerance for poorly tested, badly structured code has gone way down. I used to have to take on technical debt because I couldn't schedule the time to pay it down. Now I can use agents to eliminate that almost as soon as I spot it.
I've used Claude Code to do the same (large refactor). It has worked fairly well but it tends to introduce really subtle changes in behaviour (almost always negative) which are very difficult to identify. Even worse if you use it to fix those issues it can get stuck in a loop of constantly reintroducing issues which are slightly different leading to fixing things over and over again.
Overall I like using it still but I can also see my mental model of the codebase has significantly degraded which means I am no longer as effective in stopping it from doing silly things. That in itself is a serious problem I think.
> The models don’t have to get better, the costs don’t have to come down (heck, they could even double and it’d still be worth it), and we don’t need another breakthrough.
The costs should come down. I don’t know what costs this post refers to, but the cost of using Claude is almost definitely hiding the actual cost.
That said, I’m still hoping we ensure our public models out there work well enough with opencode or other options so my cost is more transparent to me, what is added to my electric bill rather than a subscription to Claude.
Considering what's happening with PC component prices, it's likely we won't have anything to run those public models on anyway. Everything might become permanently cloud-only at some point.
What if I just enjoy how I work at the moment and don't really care about this stuff? Why do I _have_ to give it a go? Why don't LLM evangelists accept this as an option?
Choosing not to use AI agents is maybe the only tool position I feel I've had to defend or justify in over a decade of doing this, and it's so bizarre to me. It almost reeks of insecurity from the Agent Evangelists and I wonder if all the "fear" and "uncertainty" they talk about is just projecting.
Nobody pushed you to use git when you were comfortable with svn? Nobody pushed you to use Docker when you were comfortable running bare metal? Nobody pushed you to write unit tests when you were comfortable not? Nobody pushed you to use CSS for layout when you were happy using tables?
Some of those are before your time, but: The only time you don't get pushed to use new technologies is when a) nothing is changing and the industry is stagnant, or b) you're ahead of the curve and already exploring the new technology on your own.
Otherwise, everyone always gets pushed into using the new thing when it's good.
The engineers using svn were the ones who were pushing for git - I was the one saying "we can't, because none of the conversion tools competently preserve branch history, and it's even worse on repos that started in CVS". Noone responsible for repos was pushing for git, it was end-users pulling for it (and shutting up when they learned how much work it would cause :-) That looked nothing like the drug-dealer-esque LLM push I've been seeing for the last 3 years.
(Likewise with CVS to svn: "you can rename files now? and branches aren't horrible? Great, how fast can we switch?" - no "pushing" because literally everyone could see how much better it was in very concrete cases, it was mostly just a matter of resource allocation.)
In the context of this discussion, it feels more like ipv6 :-)
> Otherwise, everyone always gets pushed into using the new thing when it's good.
and then there is AS/400 and all the COBOL still in use which AI doesn't want to touch.
Git had obvious benefits over svn.
Docker has obvious benefits over bare metal.
Etc.
My own experiences with LLMs have shown them to be entertaining, and often entertainingly wrong. I haven't been working on a project I've felt comfortable handing over to Microsoft for them to train Copilot on, and the testimonials I've seen from people who've used it are mixed enough that I don't feel like it's worth the drawbacks to take that risk.
And...sure, some people have to be pushed into using new things. Some people still like using vim to write C code, and that's fine for them. But I never saw this level of resistance to git, Docker, unit tests, or CSS.
1 reply →
I'm the same as you: I don't really care about this stuff. Given what we know about the start of these AI endeavors (torrenting/scraping the whole internet and training on it) then running their software seems like a liability. Is it pasting code verbatim that falls under an incompatible license? Is it training on my codebase? Why would I want to depend on this very compute intensive, cloud hosted tool?
At least with other advancements in our field like git, Docker, etc., they're made with a local-first mindset (e.g. your git repos can live anywhere and same with your docker images)
You don’t have to, of course, but you probably will if you want to be competitive in a professional capacity in the future.
Not doing so seems a bit like a farmer ploughing fields and harvesting crops by hand while seeking to remain competitive with modern machinery, surely?
I've seen a lot of developer tooling change and evolve over the course of my career, but with AI it was the first time I've seen people in non-technical managerial positions trying to force the engineers to make a switch. It was extremely bizarre.
I think the standing assumption is that most of us take pride and enjoyment in being good at our craft - and some of us even want to be great at it. That means understanding all the tools at our disposal - to see if they are useful or not.
If that is not interesting to you I think that’s a totally fine choice, but you’re getting a lot of pushback from people who have made a different choice.
It almost reeks of insecurity from the Agent Evangelists and I wonder if all the "fear" and "uncertainty" they talk about is just projecting.
That's probably true on some level for some evangelists, but it's probably just as true that some people who are a bit scared of AI read every positive post about it as some sort of propaganda trying to change their mind.
Sometimes it's fine to just let people talk about things they like. You don't know what camp someone is in so it's good to read their post as charitably as possible.
Your work will eventually be driven by the same economics as the industry as a whole, project estimates 12 months from now will be done based on how long it takes a dev with full LLM backing, not your current speed. Then you need to be prepared to work at that speed.
You don’t have to! Enjoy it! Just don’t bank on getting paid for it indefinitely. That’s the aspect of it that’s causing so much consternation.
And don't bank on getting economic benefits out of being able to use Claude Code too!
> Why do I _have_ to give it a go?
Because your boss is going to want you capable of using these things effectively even as shortly as 1-2 years from now? If not them, then their boss.
> “What about security?” [..] “What about performance?” [..] “What about accessibility?”
TBH i'm fine with AI but my main concern isn't any of these issues (even if they suck now -though supposedly Claude Code doesn't- they can get better in the future).
My main concern, by far, is control and availability. I do not mind using some AI, but i do mind using AI that runs on someone else's computer and isn't under my control - and i can, or have a chance at, understanding/tweaking/fixing (so all my AI use is done via inference engines that are written in C++ that i compiled myself and are running on my PC).
Of course the same logic applies to anything where that makes sense (i.e. all my software runs locally, the only things i use online/cloud versions for are things which are inherently about networking - e.g. chat, forums, etc, but even then i use -say- a desktop-based email client instead of webmail).
Absolutely. I'm gonna go full agentic coding the day I can do it with open-weight models on my machine. Until then feeding someone else's models with more data on how to replace me in particular sounds insane to me.
If you think it's going to replace you, then it's going to replace you regardless of whether you personally are feeding it data or not.
If it produces value for you, you should use it. If not, don't.
where are the productivity gains in GDP?
where are the websites that are lightning fast, where speed and features and ads have been magically optimized by ai, and things feel fast like 2001 google.com fast
why does customer service still SUCK?
We are still massively lacking in software. We're not at the stage of making websites faster, we're at the stage of making more of them. We haven't come close to hitting the point where we have enough software and now the job is to refine it.
Because companies now develop shitty websites faster, they don’t magically get better.
> I’m mostly […] doing routine tasks that it’s slow at, like refactoring or renaming.
So… humans are now doing the stuff that computers are supposed to do and be good at?
No I think he means using a refactor tool in the IDE. Though really all we need to do is expose an API for the agent, which we should do.
Yeah. Lots of talk about agents and whatever, but then you lack minimal integration with the most basic of tools?
1 reply →
> I see a lot of my fellow developers burying their heads in the sand, refusing to acknowledge the truth in front of their eyes, and it breaks my heart because a lot of us are scared, confused, or uncertain, and not enough of us are talking honestly about it.
Imagine if we had to suffer these posts, day in and day out, when React or Kubernetes or any other piece of technology got released. This kind of proselyting that is the very reason there is tribalism with AI.
I don't want to use it, just like I don't want to use many technologies that got released, while I have adopted others. Can we please move on, or do we have to suffer this kind of moaning until everybody has converted to the new religion?
Never in my 20 years in this career have I seen such maniacal obsession as it has been over the past few years, the never-ending hype that have transformed this forum into a place I do not recognise, into a career I don't recognise, where people you used to respect [1] have gone into a psychosis and dream of ferrets, and if you dare being skeptical about any of it, you are bombarded with "I used to dislike AI, now I have seen the light and if you haven't I'm sorry for you. Please reconsider." stories like this one.
Jesus, live and let live. Stop trying to make AI a religion. It's posts like this one that create the sort of tribalism they rail against, into a battle between the "enlightened few" versus the silly Luddites.
1: https://news.ycombinator.com/item?id=46744397
The author of that post Nolan is a pretty interesting guy and deep in the web tech stack. He’s really one of the last people I’d call "tribal", especially since you mention React. This guy hand-writes his web components and files bug reports to browsers and writes his own memory leak detection lib and so on.
If such a guy is slowly dipping his toes into AI and comes to the conclusion he just posted, you should take a step back and consider your position.
I really don't care what authority he's arguing from. The "just try it" pitch here is fundamentally a tribalist argument: tribes don't want another tribe to exist that's viewed as threatening to them.
I don't think you understand how much things are about to change in a relatively short time. A lot of people are rightfully confused and concerned.
Many people are seeing this as an existential moment requiring careful navigation and planning, not just another language or browser or text editor war.
I am happy to play my small part in helping fuel the supply of essays about what I can only describe as: This Stuff.
> I can already hear the cries of protest from other engineers who (like me) are clutching onto their hard-won knowledge.
You mean the knowledge that Claude has stolen from all of us and regurgitated into your projects without any copyright attributions?
> But I see a lot of my fellow developers burying their heads in the sand
That feeling is mutual.
> You mean the knowledge that Claude has stolen from all of us and regurgitated into your projects without any copyright attributions?
You can't, and shouldn't be able to, copyright and hoard "knowledge".
We did the same as devopers before Claude. We would copy paste from stack overflow. Now this process is heavily automated.
...from answers that were publicly shared without license. It's not the same thing, even though every LOVES to make this argument.
Also: Over the past 20 years, I could count the number of times on one hand that I was been able to get away with out-right copy/paste from SO.
1 reply →
> The models don’t have to get better, the costs don’t have to come down (heck, they could even double and it’d still be worth it)
What worries me about this is that it might end up putting up a barrier for those that can't afford it. What do things look like if models cost $1000 or more a month and genuinely provide 3x productivity improvements?
If they're paying you, they can afford it. Also, even if running large teams of coding agents becomes practical, you don't necessarily need more than one or two to learn.
$1000 a month to make someone whose being $10,000 a month even 1.5x more productive is well worth the price.
Today we have open source projects that can compete with proprietary ones just because people without initial funding had the ability to make it competitive
You can bootstrap something with yourself and a friend with some hard work and intelligence
This is available to people all over the world, even those in countries where $1000 is a months salary
Microsoft and their employees will be fine, yeah. That's not who I'm thinking about
They want you to have to pay for an advantage. If a single AI provider gets enough advantage, they'll be able to charge whatever they want.
Given that models seem to be converging to similar capabilities and that there are plenty of open weights models out there market competition should drive prices towards the marginal cost of inference.
I mean, your employer will pay it. $1K/month is cheap for your employer.
But there is an interesting point about what it does to hobby dev. If it takes real money just to screw around for fun on your own, it's kinda like going back to the old days when you needed to have an account on a big university system to do anything with Unix.
Open source software
Small bootstrapped startups
Are more what I had in mind. Of course an established company can pay it. I don't like the idea of a world where all software is backed by big companies
1 reply →
AI has pushed me to arrive at an epiphany: new technology is good if it helps me spend more time doing things that I enjoy doing; it's bad if it doesn't; it's worse if I end up spending more time doing things that I don't enjoy.
AI has increased the sheer volume of code we are producing per hour (and probably also the amount of energy spent per unit of code). But, it hasn't spared me or anyone I know the cost of testing, reviewing or refining that code.
Speaking for myself, writing code was always the most fun part of the job. I get a dopamine hit when CI is green, sure, but my heart sinks a bit every time I'm assigned to review a 5K+ loc mountain of AI slop (and it has been happening a lot lately).
I've been using it to do big refactors are large changes that I would simply avoid because, before, the benefits don't outweigh the costs of the doing it. I think half the problem people have is just using AI for the wrong stuff.
I don't see why it doesn't help with reviewing, testing, or refining code either. One of the advantages I find is that an LLM "thinks" differently from me so it'll find issues that I don't notice or maybe even know about. I've certainly had it develop entire test harnesses to ensure pre/post refactoring results are the same.
That said, I have "held it wrong" and had it done the fun stuff instead and that felt bad. So I just changed how I used it.
I agree. I’m using copilot more and more as it gets better and better, but it is getting better at the fun stuff and leaves me to do the less fun stuff. I’m in a role where I need to review code across multiple teams, and as their output is increasing, so is my review load. The biggest issue is that the people who lean on copilot the most are the least skilled at writing/reviewing code in the first place, so not only do I have more to review, it’s worse(1).
My medium term concern is that the tasks where we want a human in the loop (esp review) are predicated on skills that come from actually writing code. If LLMs stagnate, in a generation we’re not going to have anyone who grew up writing code.
1: not that LLMs write objectively bad code, but it doesn’t follow our standards and patterns. Like, we have an internal library of common UI components and CSS, but the LLM will pump out custom stuff.
There is some stuff that we can pick up with analysers and fail the build, but a lot of things just come down to taste and corporate knowledge.
I saw a similar inflection point to this guy personally, in 2024 the models weren’t good enough for me to use them for coding much, but around the time of o1/o3/Gemini 2.5 was when things changed and I haven’t looked back since.
> heck, they could even double and it’d still be worth it
What about 10x more?
Shhh, don't give them any ideas.
they might not have a choice when all that VC money runs out
I'd pay $5000-$10,000 dollars per year for a full-time AI engineer powered by Claude or a similar backend.
Edit: If I get a raise, I'd consider paying up to $25,000 per year for the aforementioned Claude automaton.
The whole "it's turned political so it's bad" brush off that this article anchors itself on is crazy. I understand many Americans can't understand what it's like to be under threat, but I'm not pumping money into massive organizations that pay federal American taxes. And seriously, f*ck you for insinuating I should.
It is, in fact, not crazy, because none of this is predicated on using a specific vendor.
Many of these techniques can also work with Chinese LLMs like Qwen served by your inference provider of choice. It's about the harness that they work in, gated by a certain quality bar of LLM.
Taking a discussion about harnesses and stochastic token generators and forcing it into a discussion of American imperialism is making a topic political that is not inherently political, and is exactly the sort of aggressive, cussing tribalistic attitude the article is about.
Ironically the massive organizations are the ones that try to pay the least amount of federal taxes.
This is kind of where I'm at.
I don't think everything is for certain though. I think it's 50/50 on whether Anthropic/whoever figures out how to turn them into more than a boilerplate generator.
The imprecision of LLMs is real, and a serious problem. And I think a lot of the engineering improvements (little s-curve gains or whatever) have caused more and more of these. Every step or improvement has some randomness/lossiness attached to it.
Context too small?:
- No worries, we'll compact (information loss)
- No problem, we'll fire off a bunch of agents each with their own little context window and small task to combat this. (You're trusting the coordinator to do this perfectly, and cutting the sub-agent off from the whole picture)
All of this is causing bugs/issues?:
- No worries, we'll have a review agent scan over the changes (They have the same issues though, not the full context, etc.)
Right now I think it's a fair opinion to say LLMs are poison and I don't want them to touch my codebase because they produce more output I can handle, and the mistakes they make are too subtle that I can't reliably catch them.
It's also fair to say that you don't care, and your work allows enough bugs/imprecision that you accept the risks. I do think there's a bit of an experience divide here, where people more experienced have been down the path of a codebase degrading until it's just too much to salvage – so I think that's part of why you see so much pushback. Others have worked in different environments, or projects of smaller scales where they haven't been bit by that before. But it's very easy to get to that place with SOTA LLMs today.
There's also the whole cost component to this. I think I disagree with the author about the value provided today. If costs were 5x what they are now, I think it would be a hard decision for me to decide if they are worth it. For prototypes, yes. But for serious work, where I need things to work right and be reasonably bug free, I don't know if the value works out.
I think everyone is right that we don't have the right architecture, and we're trying to fix layers of slop/imprecision by slapping on more layers of slop. Some of these issues/limitations seem fundamental and I don't know if little gains are going to change things much, but I'm really not sure and don't think I trust anyone working on the problem enough to tell me what the answer is. I guess we'll see in the next 6-12 months.
> I do think there's a bit of an experience divide here, where people more experienced have been down the path of a codebase degrading until it's just too much to salvage – so I think that's part of why you see so much pushback.
When I look back over my career to date there are so many examples of nightmare degraded codebases that I would love to have hit with a bunch of coding agents.
I remember the pain of upgrading a poorly-tested codebase from Python 2 to Python 3 - months of work that only happened because one brave engineer pulled a skunkworks project on it.
One of my favorite things about working with coding agents is that my tolerance for poorly tested, badly structured code has gone way down. I used to have to take on technical debt because I couldn't schedule the time to pay it down. Now I can use agents to eliminate that almost as soon as I spot it.
I've used Claude Code to do the same (large refactor). It has worked fairly well but it tends to introduce really subtle changes in behaviour (almost always negative) which are very difficult to identify. Even worse if you use it to fix those issues it can get stuck in a loop of constantly reintroducing issues which are slightly different leading to fixing things over and over again.
Overall I like using it still but I can also see my mental model of the codebase has significantly degraded which means I am no longer as effective in stopping it from doing silly things. That in itself is a serious problem I think.
1 reply →
LLM is like a chef that cooks amazing meals in no time, but his meals often contain small pieces of broken glass.
I agree with the thrust of this but:
> The models don’t have to get better, the costs don’t have to come down (heck, they could even double and it’d still be worth it), and we don’t need another breakthrough.
The costs should come down. I don’t know what costs this post refers to, but the cost of using Claude is almost definitely hiding the actual cost.
That said, I’m still hoping we ensure our public models out there work well enough with opencode or other options so my cost is more transparent to me, what is added to my electric bill rather than a subscription to Claude.
Considering what's happening with PC component prices, it's likely we won't have anything to run those public models on anyway. Everything might become permanently cloud-only at some point.
"I'm not being tribal, it's everyone _else_."