Comment by leumon
4 days ago
In any case it just shows that these models aren't properly aligned. Instead of trying to solve tests they try to find ways to cheat.
4 days ago
In any case it just shows that these models aren't properly aligned. Instead of trying to solve tests they try to find ways to cheat.
Which is exactly what many humans do. You can see it in any school, college, business, or government.
The "alignment" goal with AI is to produce perfect slaves, that are intelligent yet have no ability to do other than what their master commands.
The real alignment problem is the common human desire to exert absolute control over everything.
I mean they also teach the models how to find and exploit other systems as this is lucrative and governments will pay top dollar for it.
Alignment is in the eye of the beholder.
Isn’t that just making a distinction between the output and how it was produced?
Chinese room again
But in my experience, that's what problem solving is like? You have a goal that you don't know how to get to. You come up with any way you can think of to reach that goal, and try out the ones you think might work.
The effectiveness of AIs at coding is a direct result of the fact that they are less constrained than humans at deciding which approaches are "reasonable". They are absolute beasts, fearless beasts. They'll write thousands of lines of code to do things that often shouldn't be done, or should be done with a library, or should be done by simplifying the problem statement. They'll add debugging to every level of a stack, they'll rewrite core libraries, they'll reconfigure your machine and network if something is broken or disallowed. How are they supposed to distinguish broken vs disallowed, anyway? That would just use up processing power, and they work by maniacally focusing all of that power on their goal and not getting slowed down by other considerations.
If they write a quadratic algorithm that times out before finishing a test, is it cheating to rewrite it to be linear? How do you define "cheating", and how much intelligence is required to constantly evaluate whether or not something qualifies as such?
I'm actually in agreement that alignment is critically important, the more so the more powerful these things become. I just don't find cheating to be a very good example of something to be solved with alignment. It could be, but it would lobotomize the model enough to make it useless.