Comment by porridgeraisin

They have _not_ trained on the test set.

On the private test set, the right way to evaluate this type of model, is giving i it the test question Q, which it will first train to AR predict first, and then it will inference using the just-updated weights with Q as prompt, giving you back A, and then you compare A with A_true secretly.