Comment by torginus

12 days ago

It just occured to me that it underperforms Opus 4.5 on benchmarks when search is not enabled, but outperforms it when it is - is it possible the the Chinese internet has better quality content available?

My problem with deep research tends to be that what it does is it searches the internet, and most of the stuff it turns up is the half baked garbage that gets repeated on every topic.

6 comments

torginus

Reply
dsign  12 days ago

Hm, interesting. I use Kagi assistant with search (by Kagi), and it has a search filter that allows the model to search only academic articles. So far it has not disappointed. Of course the cynic in me thinks it's only a matter of time before there's so much AI-generated garbage even in academic articles that it will eventually become worthless. But when that turns into a serious problem, we will find some sort of solution (probably one involving tons of roller ball pens and in-person meaty handshakes).

Aurornis  12 days ago

> is it possible the the Chinese internet has better quality content available?

That’s a huge leap of logic.

The simpler explanation is that it has better searching functionality and performance.

The models are multi-lingual and can parse results from global websites just fine.

  • torginus  11 days ago

    Yes Im not familiar with the Chinese internet, however I've found that in expert topics, textbooks far outperform most internet content, with the sole exception of Wikipedia, which also sometimes has almost professional/academic-quality data on some topics.

    I think existence of Wikipedia is a red herring, there's no historical inevitability that people will band together to curate a high-quality encyclopedia on every imaginable topic.

    There might be similar, even broader/better efforts on the Chinese internet we (I) know nothing about.

    It also might be that Chinese search engines are better than Google at finding high quality data.

    But I reiterate - these search based LLMs kinda suck in the West, because Google kinda sucks. Every use of deep research usually ended up with the model citing the same crap articles and data you could find on Google manually, but whereas I could tell the data was no good, AI took it at face value.