Comment by artificialprint

6 days ago

Throw ARC-AGI 2 at it!

I suspect it wouldn't help too much. This model is meant for physics-based world modeling, while nearly all the problems in ARC are symbolic reasoning.

  • I'd say world modeling can provide the foundations from which symbolic reasoning can emerge, after all this is how we (humans) learn it too. There are a lot of tasks in arc that are grounded in simple physics

    • > I'd say world modeling can provide the foundations from which symbolic reasoning can emerge, after all this is how we (humans) learn it too

      As usual comparisons with humans provide little practical insight for what's achievable with ML. Humans don't have to learn everything from scratch like ML models do, you aren't expecting ML models to learn language out of a few thousands of tokens just because humans can, so similarly you shouldn't expect neural networks to learn reasoning from world interaction alone.

Yes, ARC-AGI 2 seems to game a lot of challenges that involve a (projection of) gravity and collisions, so I'd be quite interested in seeing whether it would generalize.