Comment by Daril

I recently experienced a similar situation on a client's small ecommerce site. After struggling for 4 days to find the cause of a strange excessive server load every afternoon, I discovered that a Woocommerce site was being hammered by several bots at the same time : AHrefs, Bitespider, MJ12Bot, SemrushBot, Claude, GPT, ... I set up a proper robots.txt file (the content doesn't change frequently, so indexing it every day doesn't make sense), but also set up proper rules in .htaccess to block all these user-agents. I also banned 20 or more of the IP addresses of these bots. The next step is to set up fail2ban to automatically ban them when they change IP or add additional IP addresses.