Comment by verdverm

A very simple setup, you can try here for Jev: https://jevplayground.com/

It's essentially, "predict the outcome of a coin flip", "what is the result?", and "heads" and "tails" as the answer options. With the Jev website, they do not show the "this is not a real probability" disclaimer until after you get the answer (misleading imo, should be upfront and more prominent, eg. by the input form like the gemini app)

When I give this to an LLM, it will generally respond with something like 50/50, 100% confidence.