Comment by doctoboggan

I think I might have written a comment similar to yours maybe 6 months or a year ago. I'm not quite sure to respond to these sorts of replies. I have used LLMs/Claude Code quite extensively professionally and was a very early adopter, have built tooling around LLM/agentic development, and genuinely embraced it. They aren't useless, but the short term gains you think you're getting come at a very steep price that you may not actually account for consciously for quite some time, if ever.

The skilled seniors better stop downplaying what actually led them to be skilled in the first place, and realize that the conditions to develop that skill has been gone and almost deemed unproductive in today's workplace.

Not disagreeing that LLM's are a force multiplier, but I highly doubt whatever value will end up finding multiplying in the next generation of seniors, at this rate. It's surreal to me that I have to point out that recognition AND recalling are both necessary components of skill acquisition, because humans largely knew this since the dawn of education.

I believe you are miscalculating the effect of skill atrophy, there is benefit and actual experience gained by doing the work yourself. You are an experienced dev and already have a lot of tools and knowledge under your belt so at the moment it is hard to see the actual issue, as this is just a productivity multiplierfor you. But give it a couple of years working under these conditions, your tech savvy nature will be severely diminished.

Have you really found claude to much more more capable than eg deepseek? Anthropic has little to no chance of producing a competitive business model in the long term.

Finally, someone said it. 20+ years in software and my productivity and velocity is wild right now.

> MFIC

Getting another agent to validate the first agent is a tower of sand.

> my velocity is apparent at https://github.com/pmarreck/)

Forgive me, but the active repos all look like reimplementations of existing good open source code (which of course is ideal training data) - rm_safe has rip for example. Or prototypes. Is there anything that actually has a user base > 1?

you're right, but if skills atrophy so will the efficacy of the tool.

You very conveniently avoided skill atrophy, the biggest issue for existing engineers to deal with (not even going into topic of cancelling whole hiring-junior-and-raise-senior approach which is just shortsighted and retarded to be polite, but general greed has overtaken the field so its not shocking this went out of window). Everybody using llms excessively is measurably doing worse, ie students. I have hard time believing out industry is somehow immune to that and refuse to do the experiment on myself if avoidable, which it sort of is for me.

Its like drug that will give you few years of great high, and ruin rest of your life afterwards. Use it by all means, I don't care about your output, nobody here does, you do you.

I do care about my long term skills, which aren't about piping some llm outputs. My employer ain't dumb fuck who is pushing for llms at all costs as much as possible. Anyway, most of my day work are processes, discussions, pushing things through - llms can't do a shit here, its personal conversations, connections, often psychical contact to get things done on time. Startup world would be different but I am as far from that unstable environment as I am from say gaming industry, just not worth my time outside SV area.

So I just use llms to verify my coding results, they are fine for that, but I do the creativity. Its by far the best part of my software dev work, why the heck would I be automating that away? Its like automating sex away so you can have more time... reading HN or some other way to just waste time, dumb approach from all angles.

Of course this changes if one is working on personal projects, self-employed, small startup etc but most folks here are not in that category.

I am another skilled senior, have been coding since I was 7, although you have a few more years of experience on me, and am commenting here just for the goldilocks moment, as I have read and reflected on both of your comments and find my reality is somewhere in the middle.

On personal projects, where I am in charge of all the hats (product development, UI, UX, backend, security, server admin, etc) -- absolutely crazy force multiplier. You get a nice suite of backend and e2e tests running, with full business scenario layered on top of that, and constantly running agents to do the coding, another agent on a higher level of reasoning to review that work, and sometimes occasionally poping into another competitors model to review their work just for added comfort -- it feels like wizardry. I am not vibing it, but I wouldn't say I am carefully scrolling through every line. I review whats fundamentally important, especially when it comes to data, overall structure, and large, cross cutting concerns, but I would be lying if I say some code doesn't land that I don't read. But I have the security of the test suites and validations , so I pour more effort into that.

It's a nice self reinforceing loop.

All of this might sound like I agree with you, and to some extent I do, but I am realizing as the apps I have built out like a cannon shot out of hell with tremendous speed and polish right out of the gate are starting to slow down. Feature adds are getting more complex. My memory is not what it used to be. Each run and pass through the code consumes more of my tokens and limits. I am starting to do less in the same amount of time. Codex did a vertical slice of a feature for me (well defined and well planned). It contained functionality that has historically plagued us developers -- the dreaded time. I used xHigh GPT 5.5. It had obvious bugs, but I wanted the robots to catch it. I popped it in claude (on the new sonnet 5! heyo!) -- Claude caught the bugs. Even said they "immediately stood out" I wondered how this happened. Frontier model from company A was evaluated by workhorse model from company B. All of this again took massive amounts of usage. And time.

And this is -- best case scenario, perfect world, everything is in perfect alignment.

Now for the work reality.

Multiple product and experience owners. Multiple dev teams. Different enterprise teams support services you rely on. You don't have full unfettered access to frontier models. You have to use copilot, or some other enterprise harness, and you run out of credits for the month, you are SOL. It's not as good as your claude, you think to yourself, but hey, its familiar enough, and you have 5k credits left for the month for Opus 4.8, better make the best of it. But now you burned half of them working on that Transactional Bug that was mixing synchronous and asynchronous semantics that the other guy's model should have picked up on. What happened? Maybe he didn't use Opus, maybe he used Haiku, maybe his prompt was bad. Who knows. Gotta fix it. Oh, you gotta reach across the isle and put in a request to get the Enterprise team to look at this caching inconsistency on user data that you need and is really the source of your race conditions. Tick tick tick. Model limits approaching. You start wondering if you just did all this by hand like "in the old days" would you have got it done correctly faster? Or at least, cheaper. You'll never know.

We've just done an official evaluation at work, using extensive statistics on our gigantic monorepo in a company with ~2000 devs over the course of 2 years, everyone from hardware engineers to regular old frontend engineers. It's a highly profitable and mature public company, and has been for going on a decade at this point without missing a beat. We were given infinite access & budgets to basically any and all AI tooling we could imagine, and we have several "AI Native" teams (whatever the fuck that even means). We're doing agentic coding, we have harnesses of all kind, skills, we have many teams doing spec-driven development, designers using all the various things like Figma Make and access to tools like Devin/Factory Droid/Claude Code/Codex/etc.

This is all to say, we as a company are using AI a lot in all possible corners, but thankfully our leadership isn't schizophrenic and isn't mandating everyone hit token limits or whatever, it's more of a "Let's see what works and what doesn't" type of thing, and we measure a lot of statistics. Nobody here really cares whether LLMs are the next coming of Christ or not, as a company there are many people (even in SLT) that are indifferent to LLMs, and many who are reasonably hyped.

I wish I could link to the actual document we were all shown since it has a beautiful breakdown of the methodology and a fine-grained breakdown of the stats and the categories measured, but in the grand scheme of things, ALL the AI tooling we have implemented (at least on the engineering side of the equation) has contributed to a total of... drum roll please... 7 (seven) Percent overall productivity increase! The most productive teams saw a productivity increase of around 20%, while some teams actually saw drops in productivity into the negative percentage points. My team, none of us really give a shit about AI and we're somewhere in the 3-5% range on certain categories of tasks, which I'd say is a fairly good assessment.

Productivity here is measured in many ways, including but not limited to speed of MR review and merge times, feature/ticket/roadmap closure/delivery, rollback/revert incidence rate, how often people interact with the MR review bots and implement their suggestions/fixes, how many times people check back on AI transcriptions/meeting notes (hint: Nobody looks back on any of it, it's all just noise that gets generated and never actually referenced outside a few extremely rare cases) and many more things I'm forgetting. It is an imperfect number of course, because measuring productivity in engineering is a sisyphean task, but in my opinion it is accurate to the reality on the ground and outside of all the hype and marketing bullshit.

So, I remain thoroughly unconvinced of these personal anecdotes of people being "massively" more productive, especially once you factor in the fact that we now have a 2000EUR budget/month/dev for all the AI tooling, those productivity numbers start looking pathetic once you factor in the costs (which are only increasing as the AI companies need to start recouping the gazillions they've burned). Some teams have started begging to disable coderabbit and other similar tools in their MRs because they're producing nothing but walls of noise that makes reviewing any MR a nightmare of sludging through endless slop of useless bullshit, ours included.

Btw, since this morning it's fixed another 200+ crashing test cases.

> Why even bother posting, especially as a reply to a completely unrelated comment?

If you give Anthropic money they will make your life worse in another aspect, it's relevant to all their models. The best principle is to not give money to people who want to harm you.

Anthropic has gone past fearmongering and well into terrorism. I think people on Hacker News should not recommend working with terrorist orgs.

How is that ironic knowing that an authoritarian imperialist country is leading the world in closed models ?

> We replaced all the commercial products our clients and ourselves used with our own

You’ll never guess what product your clients are looking to replace with their own next.

Dario’s own mouth https://x.com/coinbureau/status/2071330294452666695/mediavie...

https://www.judiciary.senate.gov/imo/media/doc/2023-07-26_-_... https://xcancel.com/coinbureau/status/2071330294452666695 https://www.techpolicy.press/transcript-senate-hearing-on-pr...

> "Once the weights of a model are public, they cannot be retrieved. If a model possesses dangerous capabilities, it is permanently out in the wild... We need to consider regulatory frameworks that account for the unique risks of open-source distribution of highly capable frontier models."

I'm a senior skilled developer and I find Anthropic $20 + Open AI $20 + OpenCode Go $10 offers more value than $100 on any particular service.

Juggling between all different models/agents is quite simple with Zed.

A caution about OpenCode Go though, the entire company seems to be run by AI so there's lot of billing related issues with zero support. I subscribe new every month as I lost money due to double payment with automatic subscription.

For non coding related tasks I use local models.

P.S. If anyone is interested to read more about my setup, let me know I'll publish a blog post.

I'm a heavy enough user that I have both the OAI and Anth $200 plans. I always use at least 50% of my weekly Opus quota at Extra setting (meaning I use double the limit of the $100 plan, at minimum). Max I rarely touch because it is twice as slow and the incremental capability gain is minimal. Usually if Opus can't sort something well at Extra, the answer isn't to use Max but to hand the issue off to GPT-5.5 at XHigh.

Agreed I think your strategy is optimal. This is what I landed on as well

This is actually very counterproductive with Opus 4.8 - you are wasting a lot of time.

For Opus 4.8 training with overblown internal dialogue and second opinions - Max effort burns just tokens and wastes time without much value. Spinning wheels.

In practice I don't think any harness (happy to be corrected here!) uses the lesser capability models for writing code. The cost trade-offs are rarely worth it.

They are often used for reading code though.

To expand on this, while the "big model to write a plan, small model to write the specific code" idea is quite common it trips up on edge cases.

In theory the flow works like this:

- small fast models read lots of code, and pass details to the large model to write a plan

- large model takes those details and writes a detailed plan

- medium models write the code

The issue happens when the medium model hits something that the plan didn't take into account (which happens a lot - the big model didn't actually read the code). Then it has to either guess, or pass back to the large model.

If it guesses, the plan usually starts to fall to bits.

If it passes back to the large model, inevitable the large model has to start reading lots of code. In that case you are paying the expensive tokens to read so you might as well have it write the code too (many less tokens are written than are read)

It might be possible to get this to work, but I haven't seen anyone who has tried agentic work with frontier models be satisfied with this hybrid setup.

I'd note that Amp (mentioned above) is probably the leader in using multiple providers in a coding agent but still uses frontier models to write code.

You might want to check out Amp: https://ampcode.com/

Which is your own harness and your own evals for your tasks I guess

I think that's the whole selling point of lovable?

They want to encourage diversifying model use.

Distillation attacks? Volume of calls?

I can confirm this, I too I'm not seeing much of a difference in practice

Wrong! Look at it better. It shows that Opus has superior performance but at higher cost.

Agreed. The graphs clearly show that opus 4.8 performs strictly better at the same cost per task

Yes sorry for the typo

The point is Anthropic has advertised their models in this way. There are plenty of models that can be used in long running situations that have proven to be more capable. Opus 4.8 is not that, and ironic given it's their top public model.