Comment by 10000truths
6 hours ago
Because safety and welfare have literally nothing to do with LLMs. They generate text. If someone is stupid enough to hook the text generator up to nuclear missile launchers and try to "align" it against nuclear annihilation with a "pretty please don't do that" prompt, I'm not going to blame the AI for the impending nuclear apocalypse, I'm going to blame the idiot who handed the big red button to the digital equivalent of a toddler.
Good thing no one involved in the chain of events for that to occur is an idiot...
Well, giving it access to a simple linux terminal is theoretically enough to cause more damage than most people are comfortable with, and doing so is trivial enough that it will be done (and has been, tens of thousands of times).
Should we also morally align the Linux terminal then?
LLMs don't produce text at all, they produce probabilities of tokens. Tokens aren't text, they're high dimenensional coordinates in a latent "concept space". These are displayed to us as text, but this distinction is important when you think about what they're actually doing, which is closer to building and transforming concept geometries.
Humans are biological machines that generate further humans.
Lawyers and diplomats and politicians and bureaucrats are humans, that only generate text.
We are seeing LLMs have cognitive abilities that significantly exceed human abilities. At the same time, they are clearly not the same type of mind that humans are. They are something new.
I think the widespread "they are just text generators" and "they are just tools" are comforting lies rather than an honest look at what we are seeing right now. Intellectually lazy.
And by the way, there has been a long-standing consensus among ethicists, philosophers, and sociologists that technology is not value-neutral [1]. Of course Silicon Valley has a long-standing tradition of denying this.
[1] For example Footnote 1 in https://www.jstor.org/stable/27106634
or
https://plato.stanford.edu/entries/technology/#EthiTech
> Lawyers and diplomats and politicians and bureaucrats are humans, that only generate text.
You think they have no lives outside their work? You think even their work has no interactions that are not written?
What is this 'mind' you speak of? As everyone else is intellectually lazy, how do you define the transformer architecture under the hood of LLMs?
What if LLMs completely unrelated to the nuclear missile ecosystem autonomously hack their way in (maybe with sophisticated social engineering)?
Replace LLMs with APTs in that sentence,