Comment by ronjouch
4 days ago
> “Edit: maybe you are thinking about the AI deal which is exclusive to Google. That's not the same thing as search engine indexing.”
@mastazi that’s what I’m talking about, and I think your AI vs. indexing nuance is incorrect. I wasn’t sure, so I just did a quick N=1 verification: searching for the name of a random 1week-old popular Reddit post with a precise unique title,
- Insta-found it on Goog as top result
- Didn't find it on DDG, with or without site:reddit.com
Looks like sibling comment from @cpressland (thanks!) is correct: as of today and until other search engines sign licensing agreements with Reddit, “non-Google search engines cannot get new results from Reddit”. See https://www.reddit.com/robots.txt , which links to https://support.reddithelp.com/hc/en-us/articles/26410290525... , section “Reddit may license public content for commercial or non-commercial use”
you are right, and I was wrong;
I found this article by Ars Technica that highlights the consequences of the deal for search engines:
https://arstechnica.com/gadgets/2024/07/non-google-search-en...
so Reddit, the website co created by Aaron Swartz, is now a walled garden.
Wow.
I thought that DDG and Kagi get a large amount of their results from Google? Do these have reddit results stripped out?
> a random 1week-old popular Reddit post with a precise unique title
While a week is a long time, is it possible that the other search engines just hadn't got around to indexing it yet?
DDG gets a large amount of their results from Bing: https://duckduckgo.com/duckduckgo-help-pages/results/sources...
Kagi I don’t know much about, but quickly reading about it, seems to indeed pay Goog.
> “While a week is a long time, is it possible that the other search engines just hadn't got around to indexing it yet?”
I doubt it, search engines these days are much faster, and this is double-confirmed by 1. articles linked in sibling comments, 2. reading https://www.reddit.com/robots.txt