Comment by TZubiri
17 hours ago
> Please consult me when you encounter any ambiguous edge cases
Why not check the logprobs of the output and take action when the prob of the first and second most likely token is too similar? (or below a certain threshold?
I think you're getting abstraction layers mixed up, prediction uncertainty and logical uncertainty aren't the same. In a reasoning model, it's entirely possible that there's only one likely continuation and it says something like "This edge case is ambiguous, but what the user most likely meant is X".
because this is manual? are you an llm?