Comment by supern0va
3 months ago
>I was worried this time last year that by this time this year, companies would have slashed their engineering teams down to a handful and everything would be driven by mostly autonomous agents with human guidance. But it just hasn't happened.
I find this somewhat puzzling. I thought things were moving quickly, but at this time last year I couldn't even get Claude (using Cursor) to spin me up a service skeleton that would compile, let alone do anything meaningful.
I know it feels like a long time somehow, but it was only between November and February that things started to actually somewhat work without significant hand holding. Even now, it seems like we're still figuring out how to fully leverage the current models and tooling, even in organizations that have largely gotten on board.
It's not all that surprising that people were worried and believed this. The AI companies and infrastructure companies partnering with them have spent a lot of money and time trying to convince people this is the case year after year. The critical clue people miss is that everyone claiming that has very clear financial incentives to convince people that's the case even when they know it isn't. Anyone who was actually building with LLMs and judging for themselves based on its performance knew fully well that wasn't the case year after year.
I've said this before: if anthropic (et al) thought they genuinely had a shot at replacing even 30% of white collar work, they would ABSOLUTELY NOT warn ANYONE. They would do what oil, leaded gas, and cigarette companies did. Swear under oath this is completely safe, commit GRIEVOUS societal harm that you explicitly promised wouldn't happen, and then end up in history books instead of jail for reasons beyond my ability to fathom.
No. The very fact they are trying to "warn" us means it's all marketing.
This has been corroborated for me on the engineering front that I can't find a single IC I respect who actually thought there was any evidence AI was going to live up to the hype. I saw a lot of people I always thought were idiots/sycophants/brown nosers go insane with AI. Never saw anyone id trust to help me cross a street blindfolded say more that "I may be wrong, but I'm not seeing any evidence yet".
Fwiw , you're conflating multiple things and consequently drawing premature conclusions.
It can be massively over hyped for it's current capacity and decimate the white collar work.
A lot of the difference of opinion is down to their point of view. At my dayjob, LLMs will not live up to anything because the enterprise is not structured to take advantage of it's strength. That's unlikely to change within the foreseeable future.
I strongly suspect you mostly talked with people coming from just such a background, because it's hard to go beyond our own bubbles
6 replies →
Yes, this... all the hype from the leading AI companies just pattern-matched so many past cases where things didn't pan out. really giving me the bad vibes..
>The critical clue people miss is that everyone claiming that has very clear financial incentives to convince people that's the case even when they know it isn't.
Social media is flooded with bots pushing this narrative - coding is dead, engineers are all cooked, the latest model is scarily good, "what am I for?", etc.
A good rule of thumb is that if it's a human being and not a bot, they'll use the word "slop" at some point.
coding harnesses improvements mattered more than llm improvements this past year. You could solve problems on claude sonnet on claude code that you couldn't solve a year prior.
Nope. If you use pi and its extremely minimal harness you can see a _lot_ of difference in models.
> at this time last year I couldn't even get Claude (using Cursor) to spin me up a service skeleton that would compile, let alone do anything meaningful
I've been using it to do this for 2 years now. And many people with me. The change you mention is one of is primarily one of Overton windows, of vibes.
>I've been using it to do this for 2 years now. And many people with me. The change you mention is one of is primarily one of Overton windows, of vibes.
I suspect this may have depended on the specific framework. I quite literally could not get Claude (in Cursor) to give me a basic Micronaut setup in a fresh workspace with essentially a "hello, world" API. I would guess that if you're using something like Python and FastAPI, it might have been an easier task or better represented in the data.
The difference that I observed in the Opus 4.5 era is that Claude could take a service framework it has never seen before (proprietary corp) and figure it out.
Which harness software were you using for this 2 years ago? VS Code Copilot? Cursor?
Aider for me
Very successful by just being careful and walking it forward.
Yes its about 2 years, August 2024 from git it looks like.
2 replies →
Exact same for me as WhatIsDukka.
Yeah, I strongly remember getting Sonnet 3.5 with Aider to boostrap an (albeit basic) project and getting it to work. Especially vivid because I told my roommate at the time about it and he also tried it out and was also shocked. I'd put maybe $20 in credits into the API haha. Feels so quaint, it's almost a foregone conclusion with the current models and harnesses.