Comment by papruapap
2 years ago
Do you have any source of that? I'd guess most of them just use a windows User Agent to avoid being flagged.
2 years ago
Do you have any source of that? I'd guess most of them just use a windows User Agent to avoid being flagged.
I don't and that's why I say it would be curious to see the numbers that could potentially expose the bots-vs-users discrepancy.
Without numbers an educated guess looks like this:
1) Even if say 70% of bots set Windows UA, the remaining 30% of Linux UA will still skew the numbers noticeably because 30% is much more than the "natural" Linux market share.
2) Many bots don't modify the UA just because they don't care and are not being blocked often enough, not on the domains that they scrape.
3) Many bots don't modify the UA because they care a lot and follow the strategy of emulating a real chrome desktop user with high fidelity. In this case it's better to leave the real Linux Chrome UA than to risk being detected by discrepancies between the UA and the browser capabilities detected by JS.