Comment by fn-mote
4 hours ago
> once somebody used it to develop a full exploit the LLM will start to regurgitate that information
Sure about that?
Not my understanding of how the LLMs currently work. There’s a cutoff date and no new info after that date is included in training.
Info to the contrary welcome.
Frontier models absolutely have access to current information. I just asked Claude about my fantasy football line up. It knew current injury reports, line ups and stats. Google search AI had even more information than that.
However, I’ve done some original research regarding an area that the internet has wrong and that doesn’t seem to have back propagated yet.
The models don't have access to new information. The harnesses do. All the major chat interfaces now provide the model with web search tools that they use to locate anything beyond their inherent knowledge (whether it's too new, or just something that didn't get baked in during training).
But if something hasn't been posted online publicly, they won't know about it.
> Not my understanding of how the LLMs currently work. There’s a cutoff date and no new info after that date is included in training.
You are almost certainly correct in terms of how the public models work. I guess it's possible that the labs have some tools connected for more recent stuff like this, but it seems unlikely.