Comment by cdnsteve

Yes, except we want a shared resource for an index so we aren't all crawling sites in excess.

Eg, here's the latest central or distributed DB, the site in your personal list is on it so we don't need to send the crawler there because it has already been indexed and is crawled by someone else.