Comment by timeattack

What I'm trying to say (which deviates from the initial question I've asked), is that biological brains (not just humans, plenty of animals as well) are able to not only use "random things" (whether they are physical or just in mind) as tools, but also use those tools to produce better tools.

Like, say, `vim` is a complex and polished tool. I routinely use it to solve various problems. Even if I would give LLM full keyboard & screen access, would be able to solve those problems for me? I don't think so. There is something missing here. You can say, see, there are various `tools` API-level integrations and such, but is there any real demonstration of "intelligent" use of those tools by AI? No, because it would be the AGI. Look, I'm not saying that AI would never be able to do that or that "we" are somehow special.

You, even if given something as crude as `ed` from '73 and assembler, would be able to write an OS, given time. LLMs can't even figure out `diff` format properly using so much time and energy that none of us would ever have.

You can also say, that brains do some kind of biological level RL driven by utility function `survive_and_reproduce_score(state)`, and it might be true. However given that we as humankind at current stage do not needed to excert great effort to survive and reproduce, at least in Western world, some of us still invent and build new tools. So _something_ is missing here. Question is what.