Comment by user43928
6 hours ago
You don't think it's useful to learn whether a model's "intelligence" generalizes beyond the tasks and modalities it is usually optimized for?
6 hours ago
You don't think it's useful to learn whether a model's "intelligence" generalizes beyond the tasks and modalities it is usually optimized for?
Absolutely not. Makes about as much sense as judging a car based on how good an airplane it makes.
Have you considered that the single most impressive breakthrough of LLMs as a technology is their ability to generalize beyond what they were explicitly trained on? Great analogy, pal, but LLMs aren't cars.
I disagree. If GPT-7 can draw the Mona Lisa in MS Paint via computer use, this would be interesting.
That it isn't the most efficient way to achieve the same end result is irrelevant.