Comment by Aurornis
20 hours ago
AI completely broke some of the signals management used as proxies for team competence. In the past, teams that moved quickly and had a proof of concept to show management were valued for the speed of their execution. Usually a team that could execute quickly had some competent developers who could also quickly execute on the challenges of the implementation when it scaled up to real world problems.
Now anyone with enough LLM tokens can move quickly and have a pretty proof of concept in front of management in days or maybe hours. There's a land grab to get things in front of management so you can stake your claim on something. The implementation details are assumed to be solvable by enough Claude sessions and ChatGPT prompts.
Where this falls apart is that the real difficult problems don't scale well with LLM brute force techniques. You need people who understand the problem and know how to drive the LLM agents around properly, as well as when to set them aside and go look at the code and data themselves.
The teams who try to brute force everything with LLM tokens can get a little farther every few months, but for complex problems I've seen enough variations of teams hitting a wall of complexity and tech debt with a team that doesn't know much about how their system works because they've just been asking agents to solve every problem as it comes up. With each iteration it takes a little longer for the agents to come to a solution and the system becomes a little more brittle. Even with our current mega-context-window LLMs you can reach a point where every other change to your big ball of code breaks something else, which requires another agent session to diagnose and fix, which might then break another thing down the line. The refactors to try to clean things up end up breaking even more. Having a zillion tests starts to backfire when the team can't tell which tests are testing valid results and which are baking in incorrect results.
Maybe there's an end game where agents get so good that none of this matters, but I think it's more likely that we're going to have to build up better intuitions about what good execution looks like. The old signals of moving fast are meaningly and potentially even negative now.
> AI completely broke some of the signals management used as proxies for team competence.
This is very similar to how LLMs were already breaking signals in writing. It was (is?) an enormously important proxy (however imperfect) for a bunch of things like: "put effort into this" or "is smart" or "is interested and engaged" or "has an emotion" or "has carefully considered the idea."
Heck, even basic awareness of an issue/detail gets forged, mentioned in the text but never in the mind of the sender.
In a way, LLMs have made it much more difficult to be believed. On the one hand, my inner paranoiac feels vindicated -- preparing for the worst case, over a lifetime, feels more prescient than it should. On the other hand, making decisions, specifically about how much faith to put into another unfamiliar human, becomes even less tractable.
The work can no longer speak for the human. A nightmare for the introvert.
The work, even before LLMs, struggled to speak for the introvert (and even the extrovert) -- a unique enough work usually results in puzzlement, a questioning of motives, and sometimes even overt ridicule.
Even for writing, it was a struggle. How many times did pieces of writing get ignored (or rapidly skimmed) because they were "TLDR"?
When I was very young (a teen), I wondered (after seeing the films The Matrix and Fight Club) to what degree other people were a dream-like hallucination (basically, those films made the concept of p-zombies accessible to a kid). That childish daydream, is now a reality. My inner paranoiac squeals with delight. Ah, the perversity. Even skill at deception itself can be faked, now. Even the beautiful can treated as if it were hideous.
I wonder if anyone ever made an episode of Star Trek, where one crew-member pranks another (possibly inebriated) crew-member, by having them wake up in a perfect holodeck reconstruction of their ship and crew. How long before they would notice? Weeks could go by, before a red alert dissolves the illusion.
It's after 3am, and I am not inebriated, but fuck I need to sleep. Sometimes I hope I drift into a coma, so that I can deal with things that are, on the one hand hallucinations, but on the other hand _not_ deceptions. There is a universe out there, where _their_ Matrix (film), is Vanilla Sky (amusingly, its inspiration was the film Abre los Ojos, released one year before The Matrix -- maybe in this universe, the Spanish Armada reached England).
Good night, you doomed, sad people.
This is exactly what I’m talking about. I’m seeing the same pattern. And the most useful proxy I’ve found so far for “we don’t understand the problem” is taking zero dependencies. It means that you haven’t bothered to understand what the library does and you think it can be replaced with something AI one-shots.
[dead]