Comment by simonw
18 hours ago
"It used a deprecated package"
That's because models have training cut-off dates. It's important to take those into account when working with them: https://simonwillison.net/2025/Mar/11/using-llms-for-code/#a...
I've switched to o4-mini-high via ChatGPT as my default model for a lot of code because it can use its search function to lookup the latest documentation.
You can tell it "look up the most recent version of library X and use that" and it will often work!
I even used it for a frustrating upgrade recently - I pasted in some previous code and prompted this:
This code needs to be upgraded to the new recommended JavaScript library from Google. Figure out what that is and then look up enough documentation to port this code to it.
It did exactly what I asked: https://simonwillison.net/2025/Apr/21/ai-assisted-search/#la...
>That's because models have training cut-off dates
When I pointed out that it used a deprecated package, it agreed and even cited the correct version after which it was deprecated (way back in 2021). So it knows it's deprecated, but the next-token prediction (without reasoning or tools) still can't connect the dots when much of the training data (before 2021) uses that package as if it's still acceptable.
>I've switched to o4-mini-high via ChatGPT as my default model for a lot of code because it can use its search function to lookup the latest documentation.
Thanks for the tip!
>I've switched to o4-mini-high via ChatGPT as my default model for a lot of code because it can use its search function to lookup the latest documentation.
That is such a useful distinction. I like to think I'm keeping up with this stuff, but the '4o' versus 'o4' still throws me.
Model naming is absolutely maddening.
There's still skill involved with using the LLM in coding. In this case, o4-mini-high might do the trick, but the easier answer that worry's with other models is to include the high level library documentation yourself as context and it'll use that API.
Whate besides anecdote makes you think a different model will be anything but marginally incrementally better?
> That's because models have training cut-off dates.
Which is precisely the issue with the idea of LLMs completely replacing human engineers. It doesn't understand this context unless a human tells it to understand that context.
Context7 MCP solves this. Use it with Cursor/Windsurf.