Comment by vidarh
1 month ago
> (there is no singular thread behind my comment. I think we probably have more in agreement than not, and its more a question of finding the precise words to declare the shapes we perceive.)
I moved this up top, because I agree, despite the length of the below:
> However, the current hype cycle has created expectations of reliability from LLMs that drive 'Automated Intelligence' styled workflows.
Because for a lot of things it works. Today. I have a setup doing mostly autonomous software development. I set direction. I don't even write specs. It's not foolproof yet by any means - that is on the edge of what is doable today. Dial it back just a little bit, and I have projects in production that are mostly AI written, that have passed through rigorous reviews from human developers.
The key thing is that you can't "vibecode" that. I'm sure we agree there.
There needs to be a rigorous process behind it, and I think we'll agree on that too.
Those processes are largely the same as the processes required for human developers. Only for human developers we leave a lot of that process "squishy" and under-specified.
We trust our human developers to mostly do the right thing, even though many don't, and to not need written checklists and controls, even though many do.
What is coming out of this is a start of systems that codify processes that are very much feels based with human teams. Partly because we still need to codify them for AI, but also because we can - most people wouldn't want to work in the kind of regimented environment we can enforce on AI.
Sure, there is a lot of hype from people who just want to throw random prompts at an LLM and get finished software out. That is idiocy. Even a super-intelligent future AI can't read minds.
But there are a lot of people building harnesses to wrap these LLMs in process and rigor to squeeze as much reliability as possible from them, and it turns out you can leverage human organisational knowledge to get surprisingly far in that respect.
> Because for a lot of things it works. Today. I have a setup
> There needs to be a rigorous process behind it, and I think we'll agree on that too.
I would simplify it to: “I have a setup” is the part that is doing the actual heavy lifting.
From my very unscientific survey / extensive pestering of network, the only people getting lift out of AI are people with both domain expertise/experience and familiarity with the tooling.
The types of automation I see people wanting though are fully automated customer support systems, fully automated document review - essentially white collar dark factories. (Hey thats a good term). The need is for a process that is stable, and behaves the same way every time.
It seems actual AI use cases are more like sketching - if you have enough skill you can make out the rough sketch is unbalanced and won’t resolve into a good final piece. Non experts spend far more time exploring dead ends because they don’t have the experience.
In my opinion, it’s a force multiplier for experts or stable processes, and it’s presented as Intelligence.
I feel your examples fit within these boundaries as well as the ones you have described.
I would agree with all of this. We could argue over whether/when there's sufficient intelligence for fully autonomous systems, but those systems will keep being tools for experts for the foreseeable future, and the question is just how small or large the autonomous components of that are, not whether or not you still need experts to wield them.