Comment by jsheard
2 months ago
If synthetic data is so much better then what are AI crawlers still DDOSing everyone for? Are they stupid?
2 months ago
If synthetic data is so much better then what are AI crawlers still DDOSing everyone for? Are they stupid?
Mostly. I had the "AI bot tsunami" problem on my own personal site and blocked a bunch of bot user agents in robots.txt. Most of them were from companies I had never heard of before. The only big AI name I recognized was GPTBot from OpenAI.