Comment by cheema33
10 days ago
> Average humans are pretty great at solving a certain class of complex problems that we tried to tackle unsuccessfully with many millions lines of deterministic code..
Are you suggesting that an average user would want to precisely describe in detail what they want, every single time, instead of clicking on a link that gives them what they want?
No, but the average user is capable of describing what they want to something trained in interpreting what users want. The average person is incapable of articulating the exact steps necessary to change a car's oil, but they have no issue with saying "change my car's oil" to a mechanic. The implicit assumption with LLM-based backends is that the LLM would be capable of correctly interpreting vague user requests. Otherwise it wouldn't be very useful.
The average mechanic won’t do something completely different to your car because you added some extra filler words to your request though.
The average user may not care exactly what the mechanic does to fix your car, but they do expect things to be repeatable. If car repair LLMs function anything like coding LLMs, one request could result in an oil change, while a similar request could end up with an engine replacement.
I think we're making similar points, but I kind of phrased it weirdly. I agree that current LLMs are sensitive to phrasing and are highly unpredictable and therefore aren't useful in AI-based backends. The point I'm making is that these issues are potentially solvable with better AI and don't philosophically invalidate the idea of a non-programmatic backend.
One could imagine a hypothetical AI model that can do a pretty good job of understanding vague requests, properly refusing irrelevant requests (if you ask a mechanic to bake you a cake he'll likely tell you to go away), and behaving more or less consistently. It is acceptable for an AI-based backend to have a non-zero failure rate. If a mechanic was distracted or misheard you or was just feeling really spiteful, it's not inconceivable that he would replace your engine instead of changing your oil. The critical point is that this happens very, very rarely and 99.99% of the time he will change your oil correctly. Current LLMs have far too high of a failure rate to be useful, but having a failure rate at all is not a non-starter for being useful.
1 reply →
Mechanics, and humans, are non-deterministic. Every mechanic works differently, because they have different bodies and minds.
LLMs are, of course, bad. Or not good enough, at least. But suppose they are. Suppose they're perfect.
Would I rather use an app or just directly interface with an LLM? The LLM might be quicker and easier. I know, for example, ordering takeout is much faster if I just call and speak to a person.
2 replies →
There would be bookmarks to prompts and the results of the moment would be cached : both of these are already happening and will get better. We probably will freeze and unfreeze parts of neural nets to just get to that point and even mix them up to quickly mix up different concept you described before and continue from there.
I think they're suggesting that some problems are trivially solvable by humans but extremely hard to do with code - in fact the outcome can seem non-deterministic despite it being deterministic because there are so many confounding variables at play. This is where an LLM or other for of AI could be a valid solution.