Comment by mike_hearn

Search engines track that. It's what a "long click" means. If you click a result, then return fairly fast and keep searching or clicking other links, they infer low quality (for that query at least).