Comment by bhouston
2 days ago
> This is science fiction, these models don't have access to their own weights
A bet a worm could pull along a 1GB file with weights in it and run it on a compromised machine, but luckily for us for now, 1GB isn't really enough to be really smart, yet.
We already have a 1gb model that is as capable as it will ever be, there's a proven ceiling that cannot be passed. For example: you can't make a mice-sized brain as smart as a human brain no matter how hard you try.
Please source this claim. What 1GB models are capable of has increased generation-on-generation.
> For example: you can't make a mice-sized brain as smart as a human brain no matter how hard you try.
Sure. We don't know where the ceiling is for our digital minds, though.
They have not increased in capabilities, they have increased in specialization.
If you train a small model in another domain it will begin losing capabilities in the former domain. This is effectively the sigmoid problem.
Although I will admit that if we discover a higher information density algorithm that it might change, but not by a substantial amount to where "super intelligence" in 1gb would be possible.
6 replies →
What proof of a ceiling are you talking about? Wouldn’t proving this require a good definition for intelligence, which I don’t think there is consensus on?
- https://en.wikipedia.org/wiki/Model_collapse
- https://en.wikipedia.org/wiki/Catastrophic_interference
- https://en.wikipedia.org/wiki/Fine-tuning_(deep_learning)
- https://en.wikipedia.org/wiki/Entropy_(information_theory)
As for intelligence, the only way we have that is by allowing the model to fill the blanks which have to come from the training data. The models cannot have true intelligence for as long as they are linear models, what we see with reasoning is "boxed" intelligence where the models are effectively "modifying" themselves by feeding it's own reasoning data back into input deriving most plasible output given known information. However, the model is not able to retain what it has learned therefore that intelligence is gone the moment the session is 'full'. You can go pretty far by continiously distilling discovered information, but again all that has to come from the original training data and models own outputs, which it has to take for granted as the 'intelligence' gained is lost creating what we see is the maximum possible benchmark performance and why smaller models are not able to score as high while theoretically having the same capabilities. We can see this with larger models where they can solve tasks much faster than smaller ones as it does not require to generate the solution due to the fact that the solution is already in the training data as 'baked' intelligence and it doesn't have to 'create' it during reasoning.
4 replies →
What model is that?