Comment by qarl
2 hours ago
While this is absolutely true - I'd hesitate to discount using similar agents for checking each other. Two agents will almost never hallucinate in the same way, regardless of their weights - and by having a second one (with a different context) check almost entirely eliminates the problem.
> and by having a second one (with a different context) check almost entirely eliminates the problem.
You solved one of the largest problems with current LLMs. How is it possible that nobody tried that before?
Because they do. There are already LLMs checking outputs of other LLMs, the bullshit answers that you see are the results of failures on that checks. If you remove all checks LLMs will create hallucinations even more often.
It depends what we're judging, doesn't it? If it's "is the formatting in this document compliant with our standards?" I think it's reasonable. If it's like, life-altering if it's wrong I'm less sanguine.
They have already shown algorithmic discrimination in predicting recidivism for brown people, as they are nonsensically overrepresented in the statistical data of US prison populations.
Folks should sue in a class-action lawsuit, any legal firm worth their beautiful walnut desks would seriously be happy take on that constitutionally backed mission. =3