Comment by airstrike

They had GCC to use as an oracle/source of truth. Humans intervened multiple times. Clearly writing C compilers is a huge part of its training data—the literal definition of training on test data.

Wake me up when a model trained only on data through the year 1950 can write a C compiler.