Comment by ungovernableCat
13 hours ago
If we start with the same ingredients and independently make our stews they're likely still going to taste a bit different due to the non deterministic nature of the process. Isn't it similar with training LLMs at scale? Do you get identical weights if you do multiple runs?
They don't even say what the ingredients are.
Yeah I understand that. I mean just the methods they use to acquire the training data has already had a shit ton of drama, there's no way the western labs are ever going to be public about the ingredients.