← Back to context Comment by ahmadyan 13 hours ago even in pre-training, usually 30%-50% is code these days. 1 comment ahmadyan Reply leothetechguy 1 hour ago That would be far too high in my opinion. But happy if anybody can give insights from their own experience with pretraining runs.
leothetechguy 1 hour ago That would be far too high in my opinion. But happy if anybody can give insights from their own experience with pretraining runs.
That would be far too high in my opinion. But happy if anybody can give insights from their own experience with pretraining runs.