Comment by ACCount37
1 day ago
It's social engineering reborn.
This time around, you can social engineer a computer. By understanding LLM psychology and how the post-training process shapes it.
1 day ago
It's social engineering reborn.
This time around, you can social engineer a computer. By understanding LLM psychology and how the post-training process shapes it.
No it’s undefined out-of-distribution performance rediscovered.
You could say the same about social engineering.
it seems like lots of this is in distribution and that's somewhat the problem. the Internet contains knowledge of how to make a bomb, and therefore so does the llm
Yeah, seems it's more "exploring the distribution" as we don't actually know everything that the AIs are effectively modeling.
3 replies →
I like to think of them like Jedi mind tricks.
That's my favorite rap artist!
That’s why the term “prompt engineering” is apt.
Yeah, remember the whole semantic distance vector stuff of "king-man+woman=queen"? Psychometrics might be largely ridiculous pseudoscience for people, but since it's basically real for LLMs poetry does seem like an attack method that's hard to really defend against.
For example, maybe you could throw away gibberish input on the assumption it is trying to exploit entangled words/concepts without triggering guard-rails. Similarly you could try to fight GAN attacks with images if you could reject imperfections/noise that's inconsistent with what cameras would output. If the input is potentially "art" though.. now there's no hard criteria left to decide to filter or reject anything.
I don't think humans are fundamentally different. Just more hardened against adversarial exploitation.
"Getting maliciously manipulated by other smarter humans" was a real evolutionary pressure ever since humans learned speech, if not before. And humans are still far from perfect on that front - they're barely "good enough" on average, and far less than that on the lower end.
Walk out the door carrying a computer -> police called.
Walk out the door carrying a computer and a clipboard while wearing a high-vis vest -> "let me get the door for you."
Maybe the models can learn to be more cynical.