Comment by unsnap_biceps
16 hours ago
> About 10% of the requests do not have the amazonbot user agent.
Is there any bot string in the user agent? I'd wonder if it's GPTBot as I believe they don't respect a robots.txt deny wildcard.
16 hours ago
> About 10% of the requests do not have the amazonbot user agent.
Is there any bot string in the user agent? I'd wonder if it's GPTBot as I believe they don't respect a robots.txt deny wildcard.