Comment by jeffbee
20 days ago
Ironic that there is a dichotomy between Google and Bing with orders of magnitude less traffic than AI organizations, because only Google really has fresh docs. Bing isn't terrible but their index is usually days old. But something like Claude is years out of date. Why do they need to crawl that much?
My guess is that when a ChatGPT search is initiated, by a user, it crawls the source directly instead of relying on OpenAI’s internal index, allowing it to check for fresh content. Each search result includes sources embedded within the response.
It’s possible this behavior isn’t explicitly coded by OpenAI but is instead determined by the AI itself based on its pre-training or configuration. If that’s the case, it would be quite ironic.
Just to clarify Claude data is not years old, the latest production version is up to date as of April 2024.
They don’t. They are wasting their resources and other people’s resources because at the moment they have essentially unlimited cash to burn burn burn.
Keep in mind too, for a lot of people pushing this stuff, there's an essentially religious motivation that's more important to them than money. They truly think it's incumbent on them to build God in the form of an AI superintelligence, and they truly think that's where this path leads.
Yet another reminder that there are plenty of very smart people who are, simultaneously, very stupid.