Comment by anon-3988

This is a chicken and egg problem, of course we only value and optimize for what we can do and deem anything that we can't do as unnecessary. There are things that we human simply cannot think of therefore it must not be important or does not exist.

We cannot think of anything beyond 4 dimension, so therefore there must be nothing beyond that or that things that exist in those dimension doesn't matter that much. Or more precisely, we simply cannot appreciate those things.

If we are simply trying to mimic human intelligence...well, you are going to end up with a human brain.

Suppose we have a concept X that humans simply cannot comprehend, appreciate or solve, well, why bother create an intelligence to solve that?

From this hypothesis, I personally think that any intelligence that we create will simply be an augmentation of what human desire. That is, there will always be a human part in the cog because human is the only thing can appreciate what is being created so any and all output must cater to the human involved. This will inevitably happen because we want whatever it is the human brain is doing, without doing whatever it is that the human brain is doing.

That is until we unleash a different intelligence system with agency.