← Back to context

Comment by m4rtink

3 days ago

Any way to tell if the convincing final numbers it told you are real or halucinated ?

Solve the same task with ChatGPT, Gemini and Claude. If they agree, you can be reasonably sure.

  • I'm not opposed to experimenting, but that's a a recipe for false confidence in a final decision.

    • Where they agree it shows the data supports that answer - not necessarily that it is true, where they disagree it shows you need to hedge. That's useful.

I checked them carefully myself with various other tools. It was using python to do the math so I trust it to a single standard deviation at least.