Comment by jbellis

It's not that hard to create your own uncontaminated eval for the problems you care about, and compare.

Here's mine for coding in Rust, and I definitely see the same pattern of "most chinese models do much worse than on public evals that have been around long enough to train on" that AA v5 surfaces: https://slopcop.com/power-ranking