Comment by hatthew

> only provides comparison at the highest reasoning level, which is mostly unusable for high-volume production use cases

Similarly, AFAIK evals tend to be performed at native precision, while anyone who cares about speed is probably running quantized.