Comment by sorenjan
1 day ago
I think it should be expected that a smaller model is worse at reciting memorized data than a big one. I also don't think it's a good use case of small local models. Can it find and recite Jabberwocky if given access to a web search tool?
Forgetting things isn't lossless though is it? Makes the benchmark and the finding quite suspect