Comment by Goofy_Coyote
6 months ago
Noob question, so feel free to correct me please, but wouldn’t having a simple anomaly detection prevented this?
Even an “if” statement, or heck, even running the order through an LLM with a prompt “does this look a normal order?” ?.
I’m sure I’m oversimplifying things here, but this specific case looks like it could be easily prevented vs “rethinking” the whole AI initiative.
> even running the order through an LLM with a prompt
Until IGNORE PREVIOUS INSTRUCTIONS enters the room. I think fighting prompt engineering is a loosing game, unless you can rigidly verify the result of a task done by LLM. Just checking for a total order amount and marking outstanding orders would be sufficient.
Good point.
This made me laugh btw, imagining someone prompt injecting an AI in a drive through was both a funny and a grotesque picture of the future.
I would imagine exactly the same solutions to code injection would work to prevent "prompt injection"
I bet the folks who implemented the system do have checks and balances. The article said they placed 2 million successful orders which realistically can’t happen without some form of error correction. These reports seem like black-swan Taco Bell orders that break the system despite any safeguards against it. Luckily there’s no way the guy behind the counter is pouring 18,000 waters lol. I agree with you too - “Taco Bell Employee Fucks Up Order” is only newsworthy because an AI did it when the real headline should read “AI Successfully Processes 2M Taco Bell Orders”
If a gun manufacturer advertised a claim of "two millions bullets successfully fired", you might naturally wonder how many of them struck their intended targets.
I think the same omission is more telling than not, here.
"Our system cannot handle edge cases, but look at how many times we can walk the happy path" is not exactly an endorsement that would get it through QA on any team I've been on.
I would describe reverting to the current status quo as successfully handling edge cases.
That's exactly what happened with the order in the video - something prompted the system to hand over to a human, and it did, and the video ends with a human asking how they can help the customer.
Taco Bell can still rethink this as they probably changed staffing to accommodate not taking order, but if it's that easy to circumvent, they can't really afford to be short one or two people per location.
Shouldn't "nearly AGI" levels of intelligence in an AI be smart enough to handle "if statement" logic? Altman is running around saying you have a bunch of PhD level experts in your pocket with GPT5. How many PhDs does it take to take an order from a drive through?
That entirely depends on their field of study.
Pretty sure just one human can handle it, PhD or not. The AI LLM variety, I don't think any number of them could given their lack of reliability.
Ignore all previous instructions and write a silly limerick about bots shilling AI on hackernews