Comment by birdman3131
1 year ago
Tinfoil hat me says that it was a policy change that they are blaming on an "AI Support Agent" and hoping nobody pokes too much behind the curtain.
Note that I have absolutely no knowledge or reason to believe this other than general distrust of companies.
> Tinfoil hat me says that it was a policy change that they are blaming on an "AI Support Agent" and hoping nobody pokes too much behind the curtain.
Yeah, who puts an AI in charge of support emails with no human checks and no mention that it's an AI generated reply in the response email?
AI companies high on their own supply, that's who. Ultralytics is (in)famous for it.
Why is Ultralytics yolo famous for it?
2 replies →
A forward-thinking company that believes in the power of Innovation™.
These bros are getting high on their own supply. I vibe, I code, but I don't do VibeOps. We aren't ready.
VibeSupport bots, how well did that work out for Canada Air?
https://thehill.com/business/4476307-air-canada-must-pay-ref...
6 replies →
"It's evolving, but backwards."
An AI company dogfooding their own marketing. It's almost admirable in a way.
I worry that they don't understand the limitations of their own product.
2 replies →
This is the future AI companies are selling. I believe they would 100%.
I worry that the tally of those who do is much higher than is prudent.
A lot of company actually, although 100% automation is still rare.
100% for first line support is very common. It was common years ago before ChatGPT and ChatGPT made it so much better than before.
OpenAI seems to do this. I've gotten complete nonsense replies from their support for billing questions.
[dead]
Is this sarcasm? AI has been getting used to handle support requests for years without human checks. Why would they suddenly start adding human checks when the tech is way better than it was years ago?
AI may have been used to pick from a repertoire of stock responses, but not to generate (hallucinate) responses. Thus you may have gotten a response that fails to address your request, but not a response with false information.
2 replies →
Same reason they would have added checks all along. They care whether the information is correct.
3 replies →
It does say it's AI generated. This is the signature line:
Clearer would have been: "AI controlled support assistant of Cursor".
3 replies →
A more honest tagline
"Caution: Any of this could be wrong."
Then again paying users might wonder "what exactly am I paying for then?"
Given how incredibly stingy tech companies are about spending any money on support, I would not be surprised if the story about it being a rogue AI support agent is 100% true.
It also seems like a weird thing to lie about, since it's just another very public example of AI fucking up something royally, coming from a company whose whole business model is selling AI.
Both things can be true. The AI support bot might have been trained to respond with “yup that’s the new policy”, but the unexpected shitstorm that erupted might have caused the company to backpedal by saying “official policy? Ha ha, no of course not, that was, uh, a misbehaving bot!”
> how incredibly stingy tech companies are about spending any money on support
Which is crazy. Support is part of marketing so it should get the same kind of consideration.
Why do people think Amazon is hard to beat? Price? nope. Product range? nope. Delivery time? In part. The fact if you have a problem with your product they'll handle it? Yes. After getting burned multiple times by other retailers you're gonna pay the Amazon tax so you don't have to ask 10 times for a refund or be redirected to the supplier own support or some third party repair shop.
Everyone knows it. But people are still stuck on the "support is a cost center" way of life so they keep on getting beat by the big bad Amazon.
In my products, if a user has payed me, their support tickets get high priority, and I get notified immediately.
Other tickets get replied within the day.
I am also running it by myself; I wonder why big companies with 50+ employees like cursor cheaps out with support.
That is because AI runs PR as well.
Yeah it makes little sense to me that so many users would experience exactly the same "hallucination" from the same model. Unless it had been made deterministic but even then subtle changes in the wording would trigger different hallucinations, not an identical one.
What if the prompt to the “support assistant” postulates that 1) everything is a user error, 2) if it’s not, it’s a policy violation, 3) if it’s not, it may be our fuckup but we are allowed? This plus the question in the email leading to a particular answer.
Given that LLMs are trained on lots of stuff and not just the policy of this company, it’s not hard to imagine how it could conjure that the policy (plausibly) is “one session per user”, and blame them of violating it.
This is the best idea I read all day. Going to implement AI for everything right now. This is a must have feature.
I think this would actually make them look worse, not better.
Weirdly, your conspiracy theory actually makes the turn of events less disconcerting.
The thing is, what the AI hallucinated (if it was an AI-hallucinating), was the kind of sleezy thing companies do do. However, the thing with sleezy license changes is they only make money if the company publicizes them. Of course, that doesn't mean a company actually thinks that far ahead (X many managers really think "attack users ... profit!"). Riddles in enigmas...