Comment by senko

I don't mean to say it's trivial. I'm sure there are many hard problems such as the one you mention - though that particular one is more "cleaning the index" part which might work on top of the open common corpus.

But my impression is that it's more a question of scale and engineering time than having to invent something new.

(disclaimer: I also never worked on a internet-scale search system, maybe I'm very off the bat here as well).