Comment by torben-friis
2 days ago
Also, "AI good" and "AI bad" are very silly camps to describe a topic.
You might have seen some comments by me receiving votes lately that some might classify as anti-AI. I'm not for banning it, I use it at work and at home, I learn about it.
Here a few positions I hold, for example:
- There is a fundamental deviation between what we should be seeing if we were surrounded by 100x enabled engineers and reality. We're not seeing previously untackable, complete open source projects pop up everywhere like someone coding an open-source, iphone-compatible OS in a year, or companies providing 10x more. Just POCs and small apps. LLMs have been around enough that this is pointing towards inflated claims of success, even if they are actually useful, which I'm not denying.
- LLMs provide users with a strong psychological reward (making mental workload disappear). They do so only sometimes, in a chance-based outcome. Anyone with a passing interest in psychology should realize how similar that is to the mechanics of gambling, and thus how risky it is that a user misjudges when it is reasonable to use them. Mind that I'm not saying that the tool isn't worth it, just pointing at a source of major deviation between perceived and actual outcomes that few people consider.
- There are a lot of signals that humans rely on that are broken by LLMs. "Well formatted text -> text written with careful consideration" no longer works. "Large document -> significant effort" does not hold. "Good grammar -> educated speaker" is broken as well. "decent code practices -> the PR is safeish to approve" no longer true. Some of these barriers being broken can be enablers for people, but on the whole this is going to disrupt society in fundamental and unpredictable ways.
- I think the industry is drawing unreasonable and dangerous conclusions from the advent of AI. As some commenters pointed out, if code generation is now cheap we should be seeing engineers freed to deal with non coding tasks like automated QA, user research, architecture or design, and being more able to handle bug resolution for example. We are instead seeing a push for _creating code faster_, and proposing ignoring tasks like review and quality control in pursue of speed, which is fundamentally inconsistent with speed being less of a problem. To use a flawed analogy, if your car is now 10x as fast you should be putting way more attention to how you steer, rather than asking everyone to go pedal to the metal.
- LLMs products have the potential to be extremely user hostile if enshittified. We could have probabilistic insertion of promoted material. We could have subtle political steering of people. We could have a model's performance reduced without much SLA recourse. We are not tackling those issues before they appear when it is obvious that they will appeare, and society will pay the price.
If you read with attention you'll see that no point of mine is arguing against AI usage. I don't want to bury my head in the sand and pretend LLM's don't exist or are useless. I don't want to ban them. I'm just not willing to fully allign with marketing speech and turn my brain off.
I echo almost all of your points daily and it’s incredibly refreshing to see a level-headed response like this.
Tbf I think this is probably the way most people see it at the moment from my discussions with others.
The static comes from people who have essentially staked their entire reputation on AI one way or another.
"The static comes from people who have essentially staked their entire reputation on AI one way or another."
I have a suspicion that online, those are mostly bots.
Some of it is TESCREAL ideology. Tech circles have been inundated with quasi-religious belief surrounding AI for most of the 21st century, which then biases tech enthusiasts towards extremist positions on LLMs.
Yes, the binary comments are more predictable and therefore less interesting. Mixey ones, like what you've posted here, are much better. (That probably applies to any divisive topic, of course.)
Your comment reminds me of another important variable: time. We're still at the beginning of figuring out what these tools even are, let alone how best to use them.
>We're still at the beginning of figuring out what these tools even are, let alone how best to use them.
That's a fair point, and I imagine it's going to take a long while for the dust to clear.
People are operating on semi-dark private implementations, where the approach that works today breaks tomorrow. It's like SEO, where people operate on assumptions of google's algorithm that are not provable or guaranteed to hold, so everything gets cargo culty.
Openai, anthropic et all are not fully to blame for this, I think. even if they wanted to be more transparent, it's hard to imagine what kind of determinism they could provide users with, since the product is a probabilistic emergent property more than purposeful design.
Then there's a flood of marketing and astroturfing that seeps into technical discussions, in a way that the engineering world is not really used to (other than the crypto space, that was relatively contained).
It's a really hard, multifaceted and interesting problem to solve. I just wish I saw more cold headed attempts to approach these issues scientifically.