Comment by wtallis
16 hours ago
You haven't even tried to provide a hypothetical example of what a web app should try to do using a local LLM, nor addressed the obvious questions about how that kind of thing should be standardized, what level of local LLM capability is reasonable for a web app to expect, or how permissions for that should be managed given that a local LLM is not just a tax on local storage capacity.
So why should anyone take it as a foregone conclusion that this is an instance where web devs should get what they want? In general, the browser should be acting in the best interests of the user and not automatically granting the wishes of every web site that wants to drain your battery.
One example I have that made me excited for this feature is the free recipe manager website I run.
Many of the paid-for competitors give users the ability to import unstructured recipe data these days from sites like instagram or at least text-only websites.
I can't afford to offer this as a feature since my website has no advertising and I just pay for it out of pocket, but it's an incredibly easy feature to add if you have the money to pay for tokens.
If I could use a local llm to do it though that runs in the person's own browser then I think it would definitely be valuable.
That said, I'm not sure the state of local llms provides a good enough experience yet (small models and slow) but that doesn't mean that in the future it might not be useful.
The propsosed apis do work for this purpose, albeit more slowly and lower quality
> not automatically granting the wishes of every web site that wants to drain your battery.
Pretty sure that ship sailed way back when Flash ruled the Internet, and it's still sailing more than ever today.
Browsers are just weird sandboxed VMs now. They have nothing to do with their original purpose. Don't be mad at me, I like shipping webapps that render documents server-side and use even JS incredibly sparingly. I'm just reporting what I see. The browser exists as a way to make developing completely proprietary apps with proprietary UIs for several platforms cheaper, and Chromium exists to help further that goal including, if necessary, being packaged up and shipped with those apps (Electron).
There is a link elsewhere in this comment tree addressing all of that:
https://developer.chrome.com/docs/ai/prompt-api
Google's marketing for their latest new browser feature nobody asked for shouldn't be taken at face value. Somebody outside Google needs to provide a well-reasoned assessment of the feature proposal.
And having skimmed that page, it really doesn't answer most of the important questions. Are other browsers expected to ship Google's model, or put a different model behind the API that Google has documented as being specifically for Gemini Nano?
> Somebody outside Google needs to provide a well-reasoned assessment of the feature proposal.
I'm sure that exists already, I've personally been waiting for some version of this to make it into browsers since like GPT 3.5. Every day on HN there is conversation about the tradeoffs of local vs. hosted models; the uses this API is intended for are perfectly within the capabilities of local models.
> And having skimmed that page, it really doesn't answer most of the important questions. Are other browsers expected to ship Google's model, or put a different model behind the API that Google has documented as being specifically for Gemini Nano?
Most of this is answered under the tag "Intent to Experiment" and the associated link. It's not a mandate that they're forcing on the web today, it's a public experiment intended in part to solicit feedback for a potential spec: https://groups.google.com/a/chromium.org/g/blink-dev/c/6uBwi...
1 reply →