Comment by Zambyte
1 day ago
Honestly it seems like a job for the harness, rather than the model. Sample the model with the same question, perhaps with varying temperature (?), and use that to establish a degree of confidence in the answer. If the model provides very different answers every time, respond that it doesn't know. If it responds with the same answer usually but a different answer sometimes, respond with moderate confidence. If the model always responds with the same answer, respond with certainty.
If we want to implement Intelligence, and especially now that "the box is open" we must, we can play with the "intuitive" LLM architecture to understand it and squeeze it to its potential yeld, but at some stage we have to actually implement intelligence. That implies notions of confidence and a Foundational Theory of Knowledge (knowing why you know something), among the rest (one shot learning, update through reflection etc.).
that could work sometimes but that's terribly hacky engineering