Comment by firefoxd
1 year ago
Well that was fast. Kudos, really neat. Speed trumps everything else. I only noticed the robotic voice after I read the comments.
I worked on an Ai for customer service. Our agent took the average response time of 24/48 hours to merely seconds.
One of the messages that went to a customer was "Hello Bitch, your package will be picked up by USPS today, here is the tracking number..."
The customer responded "thank you so much" and gave us a perfect score in CSAT rating. Speed trumps everything, even when you make such a horrible mistake.
"The customer responded "thank you so much" and gave us a perfect score in CSAT rating. Speed trumps everything, even when you make such a horrible mistake."
I think not everyone would react the same way. For some calling each other bitch is normal talk (which is likely, why I it got into the training data in the first place). For others, not so much.
If I'm used to waiting 2 days, and you get it down to 30 seconds you can call me what ever you want.
I'm more pissed if I'm waiting days for a response.
Me too. But I learned that not everyone is like me. And i general I also would not trust a LLM so much, that cannot divide between formal talk and ghetto slang. It will likely get other things wrong as well, humans will, too - so the error bar needs to be lower for me as a customer to be happier. I am not happy to get a fast, but wrong response and then fight for days to get an actual human to solve the mess.
2 replies →
It's also possible that it's such an unlikely thing to hear that she actually misheard it and thought it said something nicer.
Am I the only one who would be delighted to be called Bitch (or any of the worst male-specific terms) by random professionals?
"Hey fucker, your prescription has been ready for pickup for three days. Be sure to get your lazy ass over here or else you’ll need to reorder it. Love you bye"
3 replies →
Fun fact, we fixed this issue by adding a #profanity tag and dropping the message to the next human agent.
Now our most prolific sales engineer could no longer run demos to potential clients. He had many embarrassing calls where the Ai would just not respond. His last name was Dick.
I find it odd that your engineer would make the system rely on instructions (“Do this. Never do that.”). This exposes your system to inconsistencies from the instruct tuning and future changes thereof by OpenAI or whoever. System prompts and instructions are maybe great for demos. But for a prod system where you have to cover all the bases I would never rely on such a thin layer of control.
(You can imagine the instruct layer to be like the skin on a peach. It’s tiny in influence compared to what’s inside. Even more so than, in humans, the cortex vs. the mammalian brain. Whoever tried to tell their kids not to touch the cookies while putting them in front of them and then leaving the room knows that relying on high level instructions is a bad idea.)
I wonder if the solution is to run the message through another LLM to make the message as polite as possible removing any profanities. Cost >2x as much to run though.
Maybe that was their first name, at least the one they put in lol