Comment by mkozak

you may want to check out https://github.com/Forward-Operators/prr It's abandoned now as the company is gone, but back then we were trying to find an easy way to compare LLM outputs