Comment by lucfranken

It is really not that far out.

"Those iot devices are not powerful enough" is not the whole story.

Yes, big models will not run on slow gpu's and processors this year.

There are 2 things in the article: 1. Drop in a model by itself 2. Drop in a model with tools (see part: "As a sanity check")

1. Assumes that the whole "system" deployed would be stupid at the start. The article mostly describes a random "drop model on one device" and let's see how far it gets.

2. In real scenarios, the small models have cheat sheets for solving their tasks. That gives AI and the speed and performance of pre-made scripts, tools and upfront knowledge.

Add: Improve the prompt for the next device by your learnings. And there you have some flexible but also constantly improving system.

Is it slow: Yes. Does it run into many dead paths: Yes.

But a network of those devices has time, they can run 24/7.

And yes: True, Anthropic could in a financial position where there is value to post this.

And yes: It would be a wonder if scripts + flexible ai wrapper are not yet being tested in the real world.