Comment by caxap
14 hours ago
I think that vertical is actually a step function that is still limited by the human bottleneck of deciding what to build and making sure that it actually gets built. Give the AI an underspecified task, you get what you asked for and also what you didn't ask for, in good old Genie fashion.
Hill-climbing problems seem to escape this limitation. But I wonder what will happen when the AIs start acing benchmarks faster than you can write new benchmarks, and when you run out of hard problems to solve because you rely on humans formulating them.
The real vertical will come when the AI gets access to physical world instead of bash; and when the human is finally out of the loop.
Given what you write here, would you feel safe giving AI agents access to the physical world? Which kind of safeguards would you implement?
the thing to consider as well is when tooling catches up the 20m that still get it will accelerate