Comment by kypro

2 hours ago

It touches on it, but it doesn't address it.

They talk about "value alignment" but fail to define what those values are – is it aligned to the values of the US government, or are they suggesting they want to build an AI with it's own values so it can decide for itself when and how it will intervene in wars and other human affairs? And again, is an AI which disempowers humanity in this way aligned? Many would say no, although as I argue, disempowering humanity is probably better than the alternative if we can assume it's roughly aligned with our interests (which we obviously can't because it's super intelligent, but that's another issue).

Fundamentally the problem here is that humans don't have an aligned set of values you can align an AI to.

The moment you start defining what alignment actually is in practise you simply must accept it will be unaligned with the values of others. There is no getting around this and hand waving around the issue isn't good enough.

They should tell us explicitly what they're trying to build.

1 comment

kypro

Reply
indigo945  2 hours ago

No, what I mean is this part of your comment is unrelated:

    > We should stop using the word alignment. It's a BS term for a concept which  
    > simply cannot make sense if alignment is both to mean an AI which we control and  
    > an AI which will not harm us.

The article already introduces terms to distinguish between those two concepts ("goal alignment" vs "value alignment").

You are right, of course, that when we are discussing value alignment, it becomes very relevant whose values the AI is supposed to be aligned with.