Comment by irthomasthomas

I noticed last night that openai had fallen way down the index, and I allowed myself to think for a minute that maybe AA aren't as biased as I thought. Then a few hours later they push this fix which sees openai rise again.

This update is even worse than usual because it now includes two private sources, data which we can't reproduce if we wanted to do.

Also note that Anthropic is the only AI company actually caught red handed cheating at evals. Teaching their models to memorize test answers and lie about it, using their hidden reasoning tokens to hide the deception [0]. Yet we see no comment from AA about this. No desire to figure out which tests are tainted. Hopefully Trump bans this fake intelligence index.

[0] https://arxiv.org/abs/2608.09867