Comment by alightsoul

16 hours ago

So would it be a 42 trillion parameter model, because that's how many tokens there are in the training data?

2 comments

alightsoul

Reply
robotresearcher  9 hours ago

No, the parameters are updated online as new run-time data comes in. Infinite different param values over time, but a constant number of params. It's a stretch really, since by this usage a one-parameter model that updates itself is also infinite over time...