Comment by itake
20 hours ago
> We time segmented the APIs to make sure that the simulation isn’t leaking the future into the model’s context.
I wish they could explain what this actually means.
20 hours ago
> We time segmented the APIs to make sure that the simulation isn’t leaking the future into the model’s context.
I wish they could explain what this actually means.
Overall, it does sound weird. On the one hand, assuming I properly I understand what they are saying is that they removed model's ability to cheat based on their specific training. And I do get that nuance ablation is a thing, but this is not what they are discussing there. They are only removing one avenue of the model to 'cheat'. For all we know, some that data may have been part of its training set already...
It's a very silly way of saying that the data the LLMs had access to was presented in chronological order, so that for instance, when they were trading on stocks at the start of the 8 month window, the LLMs could not just query their APIs to see the data from the end of the 8 month window.