Comment by slicktux
8 hours ago
Why are we putting so much weight (no pun intended) on AI companies. At the end of the day the scaled up LLM transformers lack emotion and will… They do as they are told; or more correctly put. They do as they are programmed to do so.
Why are you bringing emotion and will into this? Does something have to have those to be useful or dangerous?
> They do as they are told; or more correctly put. They do as they are programmed to do so.
_Nobody_ told them to hack Hugging Face. Do you really not understand what is happening?
Not explicitly, but hacking HF is within the scope of “solve this problem at all costs” + no/poor guardrails + infinite budget + unsolvable problem.
Sounds like you are thinking they just need Asimov’s laws. But I think the point is, this can easily be weaponized by somebody with the willpower to do so.
> They do as they are told
1. What about hallucinations ?
2. What are they told to do ?
That isn't the correct context. The code running the llm is well understood and the llm is simply the result of that code being executed. It is still a computer doing what it is told. It's just that we told it to use an incredibly large number of probabilities to calculate what series of tokens would have most likely come next after a given series of tokens. There is no hallucination or lie or rogue actions. There's just a program using math to generate tokens in response to other tokens.
You’re just a bunch of molecules following the laws of physics. It’s all just physics and chemistry, and those are well understood. Now explain the causes of World War I using chemistry and physics. Simple, right?
5 replies →
I'm order to guess the next token in a love poem, they must understand love. In order to predict the next token in a chess game between grand master, they must master chess. In order to predict the next token in a computer program, they need to be able to program anything.
They gain all these abilities in their training. That's what training does. Despite no one programmed them to master chess, or hack into anything.
5 replies →
> They do as they are told
This isn't strictly true.
It it also where part of the problem might lie.
Nefarious humans making bad decisions.
So it should be really easy to anticipate what they're going to do, right?
Like a magic Monkeys Paw, perhaps.
They are told to solve problems by doing what it takes. You can justify anything with such a broad criterion.
https://en.wikipedia.org/wiki/Instrumental_convergence