Comment by jppope
2 months ago
I'm not sure of the legality but I definitely appreciate their product. This lawsuit seems odd because google themselves scrape content for their indexes. From what I see SerpApi is really just providing a machine interface that Google themselves refuses to provide users and visibility into SERPs which is also something that users should have available to them.
I'm probably just being naive though...
Google publishes how to control their bot - with robots.txt. They then obey those instructions. Google also takes some effort to not use all your bandwidth. Google isn't perfect, but they are at least making a "good faith" effort to be nice and this does count in court. Overall most will agree that in general what google does to allow people to find their website is worth the things that google is doing.
You can of course argue a lot of edge cases if you really want. For the most part I want to say "it isn't worth the argument". In some cases I will take your side if I really have to think about it, but in general the system google has been using mostly works and is mostly an acceptable compromise.
But their robots are enabled by default. So it is a form of unsolicited scraping. If I spam millions of email addresses without asking for permission but provide a link to opt-out form, am I the good guy?
At this point everyone knows about robots.txt, so if you didn't opt-out that is your own fault. Opting out of everyone at once is easy, and you get fine grained control if you want it.
Also most people would agree they are fine with being indexed in general. That is different from email spam where people don't want it.
4 replies →
Who says robots.txt is legally binding? Where's the Sherman Antitrust analysis?I'm more confused than before.
The courts say. With this as a long standing tradition they are likely to agree.
1 reply →
What's nice about scraping all the content for their own good while killing off websites left and right? Google needs to be sued also.
Along with all the other AI companies out there, the've committed the biggest theft in human history.