Comment by obirunda
4 hours ago
This underestimates how much of the Internet is actually compressed into and is an integral part of the model's weights. Gemini 2.5 can recite the first Harry Potter book verbatim for over 75% of the book.
4 hours ago
This underestimates how much of the Internet is actually compressed into and is an integral part of the model's weights. Gemini 2.5 can recite the first Harry Potter book verbatim for over 75% of the book.
I'm getting astrology when I search for this. Any links on this?
Iirc it's not quite true. 75% of the book is more likely to appear than you would expect by chance if prompted with the prior tokens. This suggests that it has the book encoded in its weights, but you can't actually recover it by saying "recite harry potter for me".
Do you happen to know, is that because it can’t recite Harry Potter, or because it’s been instructed not to recite Harry Potter?
2 replies →
https://arxiv.org/abs/2601.02671?hl=en-US