Comment by root_axis
2 days ago
It could be trained to say that, but it's not exactly clear how you would reinforce the absence of certain training data in order to emit that response accurately, rather than just based on embedding proximity.
2 days ago
It could be trained to say that, but it's not exactly clear how you would reinforce the absence of certain training data in order to emit that response accurately, rather than just based on embedding proximity.
Why does it seem so hard to make training data for this? You can cook up a few thousands of training data and do an RLHF.
Yes, but all that does is locate "I don't know" near the cooked up data within the embeddings. This doesn't actually reflect an absence of data in the training.
Seems easy. Have a set of vague requests and train it to ask for clarification instead of guessing.
As I said, it's possible to train it to ask for clarification, but it's not clear how to reinforce that response in a way that correctly maps on to the absence of data rather than arbitrary embedding proximity. You can't explicitly train on every possible scenario where the AI should recognize its lack of knowledge.
If the solution were easy or obvious the problem would likely have already been solved no?
We've only had ChatGPT and the like for a few years. It took Ford longer to make automatic transmissions.
3 replies →
How does it identify what's vague?
Many ways. 1) Hire some humans to label the data. 2) Let the user give you feedback. 3) Ask another LLM.