Comment by titzer
1 hour ago
So what? Give it to an agent and it will find and download the best computer chess programs available and absolutely thwomp you. I doubt you could keep a chessboard in your head with 100% precision either. Transformer models just don't have state for that.
Sure, but installing a chess program is child/teen level general ability, and playing chess well is highly trained expert level ability. Which one are we sold AI as being?
I think we are being sold AI as expert only when given tools (although that is not emphasized). The (quasi?) miracle of AI right now is that you can get an agent to accomplish the task of a team of intelligent but not exceptional humans at speeds far exceeding what the human could do. Which makes it “cheap” to throw (effectively) dozens of teams at a problem for the equivalent of hundreds of man hours.
That may not be the AI of sci fi fantasy but it’s still a game changing reality.
I often don’t see agents reaching for available or potential tools/libraries unless explicitly told to.
Sometimes they’ll even manually search or write bespoke code to search json instead of using something like jq.
[dead]