Comment by Glyptodon
1 day ago
I think this is all true, but that unlike with Moore's law and improved PC tooling and capabilities, we also have essentially existing biological evidence that there should be a way to create much better intelligent systems in terms of training, memory, and efficiency. With classic PC evolution we didn't even have that evidence but still could make a relatively strong inference (Moore's law). But here we basically have evidence that there can be something much improved and know that it's only going to take research and discovery to figure it out, not new hardware processes.
More efficient AI is possible in principle but when the next breakthrough arrives there's no guarantee that it can be implemented on current hardware architectures. Something fundamentally different may be needed, as different from current GPU/TPU chips as they are from regular CPU/FPU chips.
The existing biological evidence took billions of years of evolution to get to the state it's at. So it may not just take research and discovery but also enormous computing power.
Evolution is really slow as you say. It even stops moving at all without selecting pressure.
Yes, one might call these… evolutionary algorithms…
Exactly. Even putting aside “better”, brains show that orders of magnitude greater efficiency are possible, for training and operation.
Indeed. It feels like we're at the equivalent of what the original IBM PC offered in the personal computer revolution.
Just seeing how much has progressed as far as capability in the past 4 years as far as capability and efficiency, it's clear that there's so much more to learn and refine from.