Comment by grayhatter

Kagi uses Brave search index? huh, TIL... that's very disappointing. And it's the kinda thing that would prevent me from ever paying for Kagi. Brave's crawler, is agressive, dumb (it doesn't appear to back off if it hits a number of 503s), and critically, it ignores robots.txt. They even admit they choose to ignore it. To top that off their crawler doesn't identify itself, instead masquerading as a real browser. I've had to ban the entire Hetzner ASN from my site to get them to stop.

On one hand, I really want Kagi to succeed. They very often, do seem to care about the parts of the world and internet that I care about. But on the other... to me, willingly associating, and financing a company that willingly brags about ignoring consent, is a non-starter for me.