Comment by mellosouls

This is nice, but would be really useful measured against human performance, and yes - I understand that is likely a difficult challenge.

Many people though are going to read the headline figures and think it means - say - Opus 5 is only a quarter the strength of a human coder.