Comment by news_hacker
5 days ago
LLMs are great at knowledge transfer, the real question is how well can they demonstrate intelligence with "unknown unknown" types of questions. This model has the benefit of being released after that issue became public knowledge, so it's hard to know how it would've performed pre-hoc.
There's a long delay ("knowledge cutoff") in model training, so it probably hasn't seen the question before.