← Back to context Comment by charcircuit 5 days ago Embedded systems can make network calls to powerful, GPU equipped servers. 11 comments charcircuit Reply ceejayoz 5 days ago Sure. Claude does that. "Cogitated for 1m 50s" doesn't work for real-time applications. charcircuit 5 days ago You can submit many queries in parallel to increase throughout. Smaller models and faster hardware can reduce the time per query too. ceejayoz 5 days ago None of that gets you the 100ms response time the parent poster talked about, for something like "who is at my doorbell?" real-time uses. sebmellen 5 days ago Ok. Claude will not work for this use case because none of the sample data (weirdly blurry ID images) is in the training data. Chu4eeno 5 days ago They really shouldn't, though. charcircuit 5 days ago It can offer a ton of user value. There is a whole industry built upon this idea, Internet of Things. ceejayoz 5 days ago IoT wasn't not built on "send all the data off to a hosted GenAI". It predated them by quite a few years. 4 replies →
ceejayoz 5 days ago Sure. Claude does that. "Cogitated for 1m 50s" doesn't work for real-time applications. charcircuit 5 days ago You can submit many queries in parallel to increase throughout. Smaller models and faster hardware can reduce the time per query too. ceejayoz 5 days ago None of that gets you the 100ms response time the parent poster talked about, for something like "who is at my doorbell?" real-time uses. sebmellen 5 days ago Ok. Claude will not work for this use case because none of the sample data (weirdly blurry ID images) is in the training data.
charcircuit 5 days ago You can submit many queries in parallel to increase throughout. Smaller models and faster hardware can reduce the time per query too. ceejayoz 5 days ago None of that gets you the 100ms response time the parent poster talked about, for something like "who is at my doorbell?" real-time uses. sebmellen 5 days ago Ok. Claude will not work for this use case because none of the sample data (weirdly blurry ID images) is in the training data.
ceejayoz 5 days ago None of that gets you the 100ms response time the parent poster talked about, for something like "who is at my doorbell?" real-time uses.
sebmellen 5 days ago Ok. Claude will not work for this use case because none of the sample data (weirdly blurry ID images) is in the training data.
Chu4eeno 5 days ago They really shouldn't, though. charcircuit 5 days ago It can offer a ton of user value. There is a whole industry built upon this idea, Internet of Things. ceejayoz 5 days ago IoT wasn't not built on "send all the data off to a hosted GenAI". It predated them by quite a few years. 4 replies →
charcircuit 5 days ago It can offer a ton of user value. There is a whole industry built upon this idea, Internet of Things. ceejayoz 5 days ago IoT wasn't not built on "send all the data off to a hosted GenAI". It predated them by quite a few years. 4 replies →
ceejayoz 5 days ago IoT wasn't not built on "send all the data off to a hosted GenAI". It predated them by quite a few years. 4 replies →
Sure. Claude does that. "Cogitated for 1m 50s" doesn't work for real-time applications.
You can submit many queries in parallel to increase throughout. Smaller models and faster hardware can reduce the time per query too.
None of that gets you the 100ms response time the parent poster talked about, for something like "who is at my doorbell?" real-time uses.
Ok. Claude will not work for this use case because none of the sample data (weirdly blurry ID images) is in the training data.
They really shouldn't, though.
It can offer a ton of user value. There is a whole industry built upon this idea, Internet of Things.
IoT wasn't not built on "send all the data off to a hosted GenAI". It predated them by quite a few years.
4 replies →