Comment by moregrist

> Assume the model which initiates the worm is a high powered model.

The disconnect here is that highly powered models have billions or trillions of parameters and have compute requirements that scale with the parameter count.

If this became feasible with smaller models, we’d be in a very different world. One where you’d have your own local Fable/Opus/Astra on your desktop.

If we get there, this becomes somewhat believable. Right now it’s just too many orders of magnitude of memory and flops between the many H100s that run the current models and what smart devices have.

Your typical ESP32 chip, which is really beefy for embedded, tends to come with <1 MB of RAM and similar amounts of NVRAM. It runs two cores at a few 100s of MHz and does a bit of SIMD but it was designed for embedded IO (I2C buses, etc) not matrix multiplication. And its network speed is nothing like an Infiniband interconnect. You’re lucky getting 802.11b Wi-Fi.