Comment by Silhouette
9 hours ago
AI "support" bots that just attempt to read the published documentation for you are possibly the most annoying thing to have come out of the current AI plague.
Even Stripe - once legendary for the quality of its support - has apparently given up now. I had to deal with it recently over a case where the merchant was seeing an unexpected change in the way it was collecting payments and the AI bot was worse than useless - it actively suggested incorrect explanations and resulted in several days of trying to change the wrong things while the problem persisted.
For my own businesses we give this issue a heavy weight when choosing which services to use. We have even seriously considered moving existing integrations to different services over this one issue recently. If we're integrating with a service then we want to know there's a real person who can actually help if we have questions or anything goes wrong. Failing to provide that because it's cheaper to push everyone through the AI bot is a statement of intent about how much you value your customers.
The problem is using AI to “push” the answer to an asker. Unless the company has hidden docs they use for support (which why would that ever benefit them), I could get just as good of an answer if I point my LLM at your docs (“pull” an answer). In fact, the response might be better because I have context set up to tune it to my understanding.
Instead, you (company / support agent) have decided that I should instead have a conversation with an LLM through a worse, more opaque harness to the detriment of all of us.
It would be interesting to hear the other side, the people running support. I wonder what fraction of requests are answered by basic knowledge and stuff clearly in the docs. At some very high fraction I could see a lot of pressure/incentive to optimize for these cases.
Even more annoying, is when the integrated "chat with AI" boxes don't actually have full knowledge about the website. Tried a couple of different such boxes, and in the end I still had to crawl the website on my own to find the information.
> AI "support" bots that just attempt to read the published documentation for you are possibly the most annoying thing to have come out of the current AI plague.
Meh. It's no big change to before, where you had first-level support search for something vaguely related and just dumping a template to the user.
The thing is... it actually works better than I'd like, because in a lot of cases it turns out that you (as the user) forgot to follow one tiny step in the documentation.
What I'd actually like to see as the user is the AI actually going over my AWS account, looking up the resources and their state on its own, and figuring out what exactly I missed, but (un?)fortunately that cannot be easily done for IAM permission reasons...
The big difference IME is that first-tier support at decent companies at least had the grace to recognise when they couldn't help and needed to escalate. I can count the number of times I've seen an AI bot automatically escalate when it was unable to find the solution on zero hands.