Comment by circuit10
3 years ago
What is your definition of “reasoning” here? They are clearly able to do many things that we would call reasoning if a human did them
3 years ago
What is your definition of “reasoning” here? They are clearly able to do many things that we would call reasoning if a human did them
I mention this all the time.
I wanted GPT to give me an NGINX config for Active Collab as we're previously using Apache, and thus an htaccess file. I fed it all the documentation from Active Collab, I couldn't get anything valid out of it. Hallucinated all kinds of things that weren't there. I then gave it the URL rewrites that would be required, line-by-line, spent a long time trying to correct it. No bueno, even worse hallucination. I spent days on trying to get it to output a valid NGINX config that incorporated these URL rewrites. It can't reason, it's doing exactly what LLMs do, which is next word prediction.
I can't imagine what people are using it for in terms of a valuable addition to their workflow with how much it hallucinates. If it can't even do an NGINX config, what use is it for anything else? People saying it's helping them learn programming languages. Fuck me, they don't know when it's wrong, and it will be wrong at some point, it's an LLM.
There are flaws but also consider PEBCAK error.
For one next time when it starts hallucinating and a gentle course correction doesn't do it, just start a new chat with a different prompt approach. Having the error in its context reinforces the same mistake and sometimes it can't get out of this loop.
PEBKAC in what context? The prompts themselves or the documentation? Because I got Active Collab running on NGINX myself.
I already did this in terms of starting new chats, I spent days on it, and consulted with half a dozen devs supposedly using it in their workflows. It's very easy to make it hallucinate.
2 replies →
> but also consider PEBCAK error.
No. It's an AI error, the person between the chair and keyboard just hit enter. If this "error" goes away when I hit the enter key a few more times and get lucky, it's not my fault.
1 reply →
It’s true that their training goal sometimes rewards them hallucinations, encouraging them do that, but that doesn’t mean it’s not reasoning. If anything hallucinations involve more reasoning because they have to make up something new that wasn’t there
It's doing what LLMs do. Next word prediction. It didn't "make it up", it incorrectly predicted the relationship of words.
It isn't reasoning about the solution to a problem either, it's running as expected in relating words to each other, that doesn't mean it has any form of understanding of the words or even what it's rendering as an output.
11 replies →
>If it can't even do an NGINX config, what use is it for anything else?
Turn it around. I'm sure there are lots of people around who couldn't do it either, even with a ridiculously high amount of time to do it
Yes, but we are discussing the claim that LLMs might exhibit superhuman smartness.
1 reply →
The vast majority of humans would fail your reasoning test.
And? We are talking about AI, not humans.
People on HN claim they're using it for XYZ in development, yet it can't even generate the necessary NGINX config, despite being given the URL rewrites it'll need to incorporate.
The point is that it hallucinates. It isn't that it failed, it's that despite giving it everything it needs to know, it hallucinated all kinds of things not in the documentation, not in my prompts et al.
Why? Because it's an LLM. It isn't fit for purpose in this context. A next word prediction AI (an LLM) isn't appropriate for these kinds of problems.
2 replies →
They seem to not be able to use concepts they can "explain" (beyond having answers sufficiently dense in the training set, it seems).
Humans also take a while to learn how to use concepts that they might be able to remember an answer to, though. Also keep in mind that LLMs have a limited time to produce each token and can’t “stop to think” like a human could. Obviously LLMs aren’t as good as reasoning as humans but they clearly do some (possibly quite limited) form of reasoning
Do you have good examples of an LLM reasoning, i.e., using a concept it explains and giving an answer outside of its training data on something that has a clearly defined correct answer?
6 replies →
If you put it this way, solving a differential equation numerically also provides an answer similar to human. But you don’t call Matlab an AI.
I guess that can called a form of reasoning, LLMs are much more general in their reasoning abilities though