Comment by konschubert
1 day ago
It's the uncanny valley of AI. It's still not quite good enough yet that you can trust it blindly on a big codebase, so you still have to read and understand everything - which is often harder than just writing it up yourself.
EDIT: don’t get me wrong. I still think AI is incredibly useful for a lot of tasks! But when implementing an architecturally hairy thing, I find it less stressful and equally quick to jump down to the editor level and use AI just for code completion.
Pretty much, The one thing I use it for is as a sanity check, pretty much "Look at <SomeFile>, point out issues you see, summarise them tersely" and it'll spot stuff a code review by a human might have spotted (in the mythical land where people actually do code reviews properly and don't just flag a spelling mistake to "show they looked at it").
Beyond that I don't trust it at all and I still write all my code the meat sack way.
Trust is earned not given and it hasn't earned it yet.
If anything, I think this hype cycle is fast exposing just how many people, teams, and companies just don't care about what is correct. They just wanna feel good about themselves and get paid.
I for one welcome the fact that this whole thing has driven me back to books and deeper into the fundamentals. I have never read so much on math, hardware, and history as in the past 3 years or so.
Much the same here - I have gone back to basics and am studying a lot more than I ever did.
[dead]
Are you doing those things for your own enjoyment though, or to eventually capitalize on it?
And if it’s just for enjoyment, then doesn’t it make sense other people who want the same would just get a job where they can keep pushing things to an AI, feel good, get paid, then quickly get back to the hobbies they really love?
4 replies →
My advice is to try letting the agent fill in the gaps. You can probably architect better than it can. Write your class outlines, explicitly define the public facing bits and what you want APIs to look like. Write the key integration tests that you know ought to pass.
The real advantage is that agents routinely write code without any silly copy/paste mistakes like accidentally accessing x twice on a coordinate operation instead of x and y. You can add some comments for what the function should do, throw in some real/pseudo code and let the LLM figure it out.
> It's the uncanny valley of AI. It's still not quite good enough yet that you can trust it blindly on a big codebase, so you still have to read and understand everything - which is often harder than just writing it up yourself.
You might as well have left it with: "It's still not quite good enough yet that you can trust it". That's the core of the issue. It doesn't matter what you ask it to do, it can't be trusted. Some things are just easier to verify and correct than others.
> That's the core of the issue. It doesn't matter what you ask it to do, it can't be trusted.
I dunno about that - whenever I ask it if I'm any good, I remain confident that it will assure me that I am!
Nah, It's not even good for small code changes.
Try using it with Ansible. It spits back complete buffoonery.
I was investigating an ansible playbook yesterday that had a 45 line comment to explain a single apt install command, completely and utterly useless. I am updating my neovim to just collapse all comments, the noise is unbearable.
I've seen people report cleaner code by forbidding agents from writing comments. Anecdotal, but interesting.
5 replies →
I agree, except for the use of the word "yet" .
I think what's missing is fundamental. I think the reason it sucks so much to work with LLM-generated code is that LLMs will never "know" what it's like to be human. They don't "understand" our frustrations and motivations, and they're missing the vast array of useful mental tactics we've evolved to cope with corporal existence.
At this point I think progress towards a good colleague bot would require a new architecture which allows continuous leaning, and for the LLM to be raised as a human child (maybe in a simulation at 1000x speed or something).
I'd say it's more about learning how to organize your work more efficiently.
If you think about a product like marble: it's something that most be chiseled out of time.
Some people can chisel better products: the AI is just a better chisel.
Sometime still has to guide the chisel and judge the art/product.
In our cases, the market judges products.
I think my point is that its sometimes (!) easier to use the manual chisel rather than go for the automatic chisel and then fix its mistakes.
They say writing engages more of the brain and helps us to remember what's written more than if we just read it, or copy and paste.
When you say it's easier to go manual, it seems you're talking about learning retention. And you're right.
But seniors have learned enough that they're able to iterate quickly with AI.
They know how to organize their work, manage change, tasks. They know how to break a problem down into smaller pieces. They're aware of context windows, token cost, estimated task lengths, etc. And most importantly, and to your point about ease: they have less to learn so retention isn't an issue.
I have no opinion about whether we're in a good or bad situation, just making arguments from the toilet really.
2 replies →
> In our cases, the market judges products.
It happens, but it's rare. When last did a product take over a market without 100s of millions, sometimes even billions, of VC dollars?
There is no motivation to build a better mousetrap today, because the drooling idiot with a Claude account will look at how quick you signup users, clone it in a week (hey, it only needs to be superficially the same), and get VC money to dump until you go out of business.
This seems to misunderstand the parent comment. The sibling comment plays along with the analogy and mentions having to fix the mistakes of an automatic chisel, but the original comment is alluding to the fact that it is definitely not an automatic chisel. There is no uncanny valley in an automatic chisel.
Programming up until this point was done by using deterministic tools to build products. LLMs appear to be nondeterministic tools in their current incarnation, at least to humans.
If an automatic chisel had a feature that could switch from chiseling from sedimentary rock to chiseling marble but would randomly and nondeterministically switch to the other mode during its use it would be considered defective. But with LLMs the industry has collectively decided that the nondeterministic automatic chisel builds so fast that the current defect rate from the nondeterminism is acceptable.
It would make sense to isolate the last line of defense from LLMs, ie the tests, but this rarely seems to happen any more. Once the tests are contaminated with LLMs all bets are off.
And people forget that along with atrophying skills and reasoning due to less coding, the skill degradation is hastened because the programmer is faced with the reality that they would have to constantly figure out, review or edit someone else's code (ie, the LLM's) if they truly wanted to maintain a last line of defense. But as this type of task is literally the least liked task in programming, the programmer passes it off to the LLM as well to avoid burnout...
this is my daily life now and its miserable
its an overload spiral that gets worse at each iteration (even while 'gardening' our agents/skills)
the worst part is using the llm to generate the specs/features; its shit from the start and tumbles downhill from there...
I had a huge post written up to address your points, but I have to get moving, my ass is literally falling asleep.
It's easier just to say: It's all about time, my man. Nobody wants to waste it.
1 reply →
I think the speed/context size of the large models is a threshold. I've been using a local model and watching it do killer stuff, and also shit out useless things; all in real time, requiring active steering.
Your assumption is that LLMs will ever leave this uncanny valley.
Maybe unforeseen breakthroughs and different architectures are achieved. Given LLM fundamental shortcomings grounded in mathematics and information theory, I highly doubt they will and we will always need to deal with these issues in some capacity.
I think that bigger context windows help, but I feel that for AI to cross this chasm, it needs to be able to encode more abstract context knowledge. I think the holy grail here is online learning.
Agreed. For AI to be something you can trust to operate autonomously, it needs to actually be able to understand the things it is working with and reason about them. LLMs cannot, by their very nature, do that. There can be no reliability with such a tool.
[dead]