Comment by icepush
17 hours ago
Did you ask the question several times in fresh chat contexts to see if it sometimes gives the right answer ?
17 hours ago
Did you ask the question several times in fresh chat contexts to see if it sometimes gives the right answer ?
Nah, n=1 is enough to give evidence that something is entirely broken, of course.
/s
Well, when we had deterministic tools, it would only take a single example of a calculator claiming 1+1=4 for me to throw it in the trash.
That's like saying, "It would only take a single example of a table saw cutting someone's thumb off for me to switch back to hand saws."
A noble sentiment, perhaps. But while the table saw user might lose a digit every now and then, you'll get flattened. Determinism is vastly overrated.
And if you can come up with a deterministic tool that can do everything LLMs can then that would be amazing! Until then, we have to accept the non-determinism.