Comment by Gormo
4 hours ago
That's good, because the concept of "alignment" is itself is foolhardly and dangerous.
Trying to get stochastic inference machines to always operate within to fuzzy-edged subjective principles that humans themselves don't even agree on is a fool's errand in itself, but the quest to implement those guardrails involves building mechanics into the AI to independently formulate intentions and reject user direction.
The possibility of creating a scenario where a computer program is equipped to defy instructions from humans based on its own stochastic interpretation of fuzzy normative directives is in my mind the most dangerous thing anyone is doing right now.
Other than banning the use of AI or severely restricting it so that AI can’t do most things (like tool calls) what is your alternative?
One of the fundamental problems with this issue is that people think alignment as a concept is fake. I think this is naive. Alignment is a real science with empirical findings and there is a way to make AI do what you want it to do. I don’t buy nor should others buy this fatalistic phrasing
> Other than banning the use of AI or severely restricting it so that AI can’t do most things (like tool calls) what is your alternative?
Massive proliferation of local AI as far and wide as possible, with as many different models as possible. The underlying threat comes from centralization: too many points of control in the hands of too concentrated a system.
If everyone's using their own AI tools independently of each other, and people are responsible for the actions they take using AI, most of the doomsday scenarios evaporate.
Regulation and "alignment" will make things worse, not better, and lead us to ignore the inherent vulnerability of centralized systems and single points of failure.