Comment by lyu07282

The question I have is if this is really generalizing, this "central preference vector" seems to exist as this work shows, but was that vector just the result of OpenAIs RLHF dataset and constrained to the examples they used? Since we don't have access to that dataset we can't say for sure(?). But perhaps it doesn't matter?