Someone deleted a reply to this comment “And where does ChatGPT get the data for those answers?” they wrote and I think the question is important: LLM chatbots can crawl the web just as well as Google.
Nah, just another startup that’ll get acquired or a similar fate. Building the capability is a superior option imho, if you’re at Anthropic or OpenAI scale. Cut out the middleman.
Someone deleted a reply to this comment “And where does ChatGPT get the data for those answers?” they wrote and I think the question is important: LLM chatbots can crawl the web just as well as Google.
The answer is Exa
Nah, just another startup that’ll get acquired or a similar fate. Building the capability is a superior option imho, if you’re at Anthropic or OpenAI scale. Cut out the middleman.
1 reply →
Huh? But only if you give it a tool it can use - which brings you back to Google (or Bing)
Anyone can leverage common crawl and/or run their own crawler, to do so isn’t novel.
https://commoncrawl.org/