Comment by leothetechguy 16 hours ago this is the rl run, not the pretraining run 2 comments leothetechguy Reply ahmadyan 15 hours ago even in pre-training, usually 30%-50% is code these days. leothetechguy 3 hours ago That would be far too high in my opinion. But happy if anybody can give insights from their own experience with pretraining runs.
ahmadyan 15 hours ago even in pre-training, usually 30%-50% is code these days. leothetechguy 3 hours ago That would be far too high in my opinion. But happy if anybody can give insights from their own experience with pretraining runs.
leothetechguy 3 hours ago That would be far too high in my opinion. But happy if anybody can give insights from their own experience with pretraining runs.
even in pre-training, usually 30%-50% is code these days.
That would be far too high in my opinion. But happy if anybody can give insights from their own experience with pretraining runs.