Comment by Root_Denied
1 day ago
I'm seeing this happen in the security space right now. Someone on my team I was helping train and bring along is all of sudden regressing in their understanding of the issues we're working on, and instead focusing on AI tool outputs to do their job for them.
I can share a weird story:
Usually, I take my time to understand each keyword of the code I'm looking at, especially if it is new to me, like terraform.
I work in a team/with one architect, who only did the DevOps/Infra stuff for the past years and I had the expectation he knows what he is doing and talking about.
At around 2 weeks, I noticed how his knowledge has severe gaps and how he takes things at face value or uses terminology interchangably, which confuses me. It sounds plausible, but it does not actually translate into a working system or shared understanding.
Then one day I did some pair programming with him and whenever there was an error or a resource missing, he would type it into the LLM, copy paste it out of it and then brute force error messages. He did not even wait a second to think or reconcile whats happening on the screen or what the exact requirement is. Never taking one step back and questioning any assumption.
Now that the timeline is shifting and everyone starts to be stressed, he continues to vibe code through me and it is so tiring, there is no higher level planning or architecture, its just a reactive type of trial and error to be faster. It feels like these people are so used to talking to bots, that they treat you like an agent they can chat to or talk through monologs with.
It is quite shocking how people went from being humble (learn the basics or close the gaps in understanding) to full on authority on everything and berating people 24/7...
So right now I'm considering quitting IT for a couple of years until people calm down, but I think its pretty futile
I don't think it's generalizable. The kind of person who copy and pastes from the AI is the kind who did the same from StackOverflow before. It's more compelling, and we probably see more of them because of it, but it's the same general thing.
The kind of person who insists on understanding things and working through the problem has always been rarer. It's not "humble", it's "inquisitive" and "persistent".
I'm seeing people who _used to be_ like that losing that understanding without realizing it's happening - they have a superficial idea of what the code is doing, enough to feel like they understand it, but the change is apparent when watching them handle something unexpected.
6 replies →
> I don't think it's generalizable. The kind of person who copy and pastes from the AI is the kind who did the same from StackOverflow before. It's more compelling, and we probably see more of them because of it, but it's the same general thing.
There's an infinity between someone who has to panel beat what they copied off SO and someone who just bangs their head repeatedly against an LLM.
In essence, you are correct. What I see with LLMs is how they will happily make up stuff, meanwhile you can pretty much exhaust SO, until you are forced to understand the problem you are having.
You don't have to be a certain person to fall into this trap, you just condition your brain to accept this workflow somewhere.
Pair this with day to day work stress like time and amount of tasks and you almost give in to a sort of addiction to deal with it all.
> The kind of person who copy and pastes from the AI is the kind who did the same from StackOverflow before. The kind of person who insists on understanding things and working through the problem has always been rarer.
Well, this is where I disagree. I have coworkers who used to insist on understanding and now are doing exactly as OP wrote - copy/pasting from the LLM to brute force error messages. I don't know if it's generalizable but this is what I see in <big tech> working on a frontend team with mid to senior level engineers who I respect.
One example a few weeks ago, I was helping a coworker root cause a bug in a React codebase. Pair programming isn't necessarily common but sometimes you see someone banging their head at a problem and you get curious. It turned out there was an effect (a callback that's invoked whenever some state changes) that invokes an API and this effect caused an infinite render loop because the error handling wasn't written correctly. It was something silly like - Call this API if we have no data -> get error -> update state -> call API again because we have no data...
That was almost immediately what I suspected but my colleague was pulling out all the stops usually reserved for when you're desperate or need a sanity check, like logging to stdout after each line. Both my colleague and the LLM were convinced the problem lied in the pagination logic of the helper that invokes the API. He ended up rewriting that helper imperatively and functionally. To his credit, he rewrote it by hand and implemented the recursion correctly but he was baffled when the problem remained. Completely surprised Pikachu face.
I don't think my coworker changed or suddenly stopped caring. It seems much more likely this is a predictable outcome when you lean heavily into AI authoring code for a sustained period of time. I'd also say that in itself is a consequence of the extreme pressure being exerted across the entire company to ship more code and review more code, faster.
The difference between the two is that friction is learning.
the less friction the less growth.
So even if they copied from SO they would have better knowledge than an Ai user.
This isn’t speculation either, theres an MIT study which this is based on. https://www.media.mit.edu/publications/your-brain-on-chatgpt...
2 replies →
Great comment.
At my place, this is what they want. They want people to smash through things as fast as possible. They don’t want people to sit and craft a solution which takes in to account the whole. They are choosing tools which are low code, and use llm’s to produce what they need. as they say “this is the way things are going”.
Even that would be acceptable. But they still blame the programmer, who was pressured into this, for any fallouts because of it. Basically trying to have it both ways. The unreasonableness of it is infuriating.
2 replies →
In today's episode:
"Programmers frustrated that they're being treated like craftspeople or delivery drivers or factory workers, etc, etc."
18 replies →
> It is quite shocking how people went from being humble (learn the basics or close the gaps in understanding) to full on authority on everything and berating people 24/7...
That was never humility. That was fear. This technology is just enabling people to show their through colors. The unearned confidence is a hell of a drug.
You captured this phenomenon very well in this comment. Appreciate you sharing it because it’s hard to describe exactly what makes this sort of behavior so bizarre.
@backlava12 fyi your comments are all dead in this thread
1 reply →
[dead]
I don't think this is going to change because it helps automate code production. As in, once we reach a point where this method produces acceptable results, we can fire all engineers and automate the process.
>> he continues to vibe code through me
So quit pair programming. I never did, never will do that, nor worked at a place that remotely encouraged that. Each to their own, that's how it should be.
I wish this was always possible, but sometimes you do it when stuff is ambigous or the task is almost undefined and you just can't reach far enough for someone to fix it, so going over it and talking is simpler
In my mind, this, not copyright or water use, is the best reason to boycott AI. It'll make you incompetent.
I saw this the past year - new employees would put problems into Claude first instead of debugging. A year back I was debug manually first, now I do the same.
The speed AI debugs at is incredible and yes, we lose touch the more we use it like any manager feet up barking orders to their underlings to get things done.
At least at my company the OKRs are quite clear and demand heavy AI utilization above all else
To be fair the cyclical nature of funding and defunding security teams which causes scaling up and scaling down, has always left a race to the bottom in security.
Any CISO or head of security loves the ideal of completely using AI to handle incidents, tune detections, implement mitigations, track vulnerabilities, pen testing, etc. This feels like it driving security teams to have less critical thinking.