Comment by lesam
4 days ago
So you want the probability that the answer is correct, but Jev is providing the probability that its answer is optimal?
4 days ago
So you want the probability that the answer is correct, but Jev is providing the probability that its answer is optimal?
Yes, and TypeSafe itself says Jev returns "calibrated probabilities", which is the former.
From TypeSafe docs:
"Higher probability should correspond to a greater chance that the answer is correct."
"Outcomes assigned a probability of 0.2 should occur about 20% of the time."
https://docs.typesafe.ai/introduction/machine-learning-prime...