Comment by meowface

The current pace of advancement and the fact that fresh writing data will likely not be a requirement at all to continue accelerating. The pretraining paradigm no longer needs this.