Comment by HPsquared
15 hours ago
They are a cooked stew; the ingredients have already been chopped and mixed together. You can add things, yes, but you can't inspect the ingredients.
15 hours ago
They are a cooked stew; the ingredients have already been chopped and mixed together. You can add things, yes, but you can't inspect the ingredients.
If we start with the same ingredients and independently make our stews they're likely still going to taste a bit different due to the non deterministic nature of the process. Isn't it similar with training LLMs at scale? Do you get identical weights if you do multiple runs?
They don't even say what the ingredients are.
Yeah I understand that. I mean just the methods they use to acquire the training data has already had a shit ton of drama, there's no way the western labs are ever going to be public about the ingredients.