Comment by fooker
17 hours ago
I'm claiming that the max amount of intelligence you get out of a 100GB model is unlikely to be that much lower than what you can get out of a 1TB model.
17 hours ago
I'm claiming that the max amount of intelligence you get out of a 100GB model is unlikely to be that much lower than what you can get out of a 1TB model.
That depends on whether you can separate intelligence from memorization. I think we can agree that theres more than 1 TB of memorizable information in the world, even when highly compressed.
Sure, agreed.
The question then is, how much memorization is really needed for intelligence if you can query structured information?