Comment by syndeo

I for one run a small Scripture-study web app that makes use of this on Chrome browsers when available to provide summarizations of long commentary articles. I'm also looking to use it to power topical search.

I allow free open access to the content, as blocking it behind an account signup doesn't sit right ethically—it should be open and free for everyone.

The issue there is that there's no way to easily secure the API from being hammered by bad actors. (Due to the often-controversial response many have to Scripture, apps like these draw special kinds of negative attention.) You can set rate limits, but people can still abuse those, just to try to burn your money. I can get by for free (or relatively cheap, fully paid out-of-pocket as none of this is monetized) on Vercel/Netlify/etc, but inference is expensive, and a prime target for those trying to cause trouble.

If in the future the web exposes local "foundation" models that web apps can assume are present, that would open up great possibilities like these for indie devs like myself. Being able to offer useful and compelling features without worrying about abuse would be nice. That's my point.