Comment by viraptor

Here it matters much less than in generic LLMs though. There's no chance of test set leakage since the network is not general purpose / not trained on the internet.