Comment by energy123
12 days ago
The "singularity" can be decomposed into 2 mutually-supportive feedback loops - the digital and the physical.
With frontier LLM agents, the digital loop is happening now to an extent (on inference code, harnesses, etc), and that extent probably grows larger (research automation) soon.
Pertinent to your point, however, is the physical feedback loop of robots making better robots/factories/compute/energy. This is an aspect of singularity scenarios like ai-2027.
In these scenarios, these robots will be the control mechanism that the digital uses to bootstrap itself faster, through experimentation and exploration. The usual constraints of physical law still apply, but it feels "unbounded" relative to normal human constraints and timescales.
A separate point: there's also deductive exploration (pure math) as distinct from empirical exploration (physics), which is not bounded by any physical constraints except for those that bound computation itself.
> With frontier LLM agents, the digital loop is happening now to an extent
I see no evidence of this, just a lot of people claiming it (very loudly, for the most part).
> that extent probably grows larger (research automation) soon
The word probably is doing a lot of work here.
> The usual constraints of physical law still apply
There are knowledge constraints, too. I can't build a quark matter processor without understanding quark matter to a vastly higher level than we do now. I can't do that without experiments on quark matter, I can't do experiments without access to a lot of energy, material, land, &c, that need to be assembled. There are a huge number of very difficult and time-consuming instrumental goals on the path to fundamentally better compute.
> A separate point: there's also deductive exploration (pure math) as distinct from empirical exploration (physics), which is not bounded by any physical constraints except for those that bound computation itself.
Sure, but physics requires math that is definitionally applied, not pure, and engineering requires physics.