Comment by _the_inflator
5 days ago
Exactly, and all on an embedded system with quite restrictive settings and no overclocked Intel lastest generation combined with NVIDIA's 10k graphic cards.
5 days ago
Exactly, and all on an embedded system with quite restrictive settings and no overclocked Intel lastest generation combined with NVIDIA's 10k graphic cards.
Embedded systems can make network calls to powerful, GPU equipped servers.
Sure. Claude does that. "Cogitated for 1m 50s" doesn't work for real-time applications.
You can submit many queries in parallel to increase throughout. Smaller models and faster hardware can reduce the time per query too.
2 replies →
They really shouldn't, though.
It can offer a ton of user value. There is a whole industry built upon this idea, Internet of Things.
5 replies →