With multiplication the question makes sense due to the commutative property but division does not have that so the question becomes ambiguous... And now I see that the model even points this out.
There is no ambiguity, the problem is that three numbers, divided together, without the order specified, must be equal to their sum.
You can find solutions for a / b / c, or b / c / a, or c / a / b, any combination of them and the solution will be correct according to the problem description.
Besides, what's does it even has to do with it concluding with confidence:
"The fundamental issue is that division tends to make numbers
smaller. It's mathematically impossible to
find three numbers where these operations result in the same value."?
> You can find solutions for a / b / c, or b / c / a, or c / a / b
This is a clear case of ambiguity.
Even the classic question is ambiguous: "Which 3 numbers give the same result when added or multiplied together?"
Lets say the three numbers are x, y and z and the result is r. A valid interpretation would be to multiply/add every pair of numbers:
x * y = r
y * z = r
x * z = r
x + y = r
y + z = r
x + z = r
However, I do not think that this ambiguity is the reason why OpenAI o1 fails here. It simply started with an untractable approach to solve this problem (plugging in random numbers) and did not attempt a more promising approach because it was not trained to do so.
Does the commutative property change anything here? A, B and C are not constrained in any way to each other, so they can be in whatever order you want anyways...
Moreover, addition is commutative so it doesn't matter what order the division is in since a/b/c = a+b+c = c+a+b = ...
So I'd say that the model pointing this out is actually a mistake and it managed to trick you. Classic LLM stuff: spit out wrong stuff in a convincing manner.
Order doesn't matter with multiplication (eg: (20 * 5) * 2 == (5 * 2) * 20) but it obviously does with division ((20/5)/2 != (2/5)/20) so the question doesn't make sense. It's you making grade-school level mistakes here.
The question makes perfect sense. Here it is written in logical language. I'm curious at which point does it stop making sense for you?
numbers divided together
↓----------↓
((a / b / c) = a + b + c) ← numbers added together
| ((a / c / b) = a + b + c)
| ((b / a / c) = a + b + c)
| ((b / c / a) = a + b + c)
| ((c / a / b) = a + b + c)
| ((c / b / a) = a + b + c)
| ((a / (b / c)) = a + b + c)
| ((a / (c / b)) = a + b + c)
| ((b / (a / c)) = a + b + c)
| ((b / (c / a)) = a + b + c)
| ((c / (a / b)) = a + b + c)
| ((c / (b / a)) = a + b + c) = true
With multiplication the question makes sense due to the commutative property but division does not have that so the question becomes ambiguous... And now I see that the model even points this out.
There is no ambiguity, the problem is that three numbers, divided together, without the order specified, must be equal to their sum.
You can find solutions for a / b / c, or b / c / a, or c / a / b, any combination of them and the solution will be correct according to the problem description.
Besides, what's does it even has to do with it concluding with confidence: "The fundamental issue is that division tends to make numbers smaller. It's mathematically impossible to find three numbers where these operations result in the same value."?
> There is no ambiguity
Yet you give three different interpretations:
> You can find solutions for a / b / c, or b / c / a, or c / a / b
This is a clear case of ambiguity.
Even the classic question is ambiguous: "Which 3 numbers give the same result when added or multiplied together?"
Lets say the three numbers are x, y and z and the result is r. A valid interpretation would be to multiply/add every pair of numbers:
However, I do not think that this ambiguity is the reason why OpenAI o1 fails here. It simply started with an untractable approach to solve this problem (plugging in random numbers) and did not attempt a more promising approach because it was not trained to do so.
1 reply →
Does the commutative property change anything here? A, B and C are not constrained in any way to each other, so they can be in whatever order you want anyways...
Moreover, addition is commutative so it doesn't matter what order the division is in since a/b/c = a+b+c = c+a+b = ...
So I'd say that the model pointing this out is actually a mistake and it managed to trick you. Classic LLM stuff: spit out wrong stuff in a convincing manner.
Order doesn't matter with multiplication (eg: (20 * 5) * 2 == (5 * 2) * 20) but it obviously does with division ((20/5)/2 != (2/5)/20) so the question doesn't make sense. It's you making grade-school level mistakes here.
The question makes perfect sense. Here it is written in logical language. I'm curious at which point does it stop making sense for you?
So you want it to solve 12 simultaneous equations? LLMs are not good at that. Is there in fact an answer? ChatGPT says no.
https://chatgpt.com/share/66e482cc-331c-8013-98ca-999d7d3f3e...
1 reply →