Comment by storystarling

For bootstrapped GenAI apps, moving inference to the browser is basically an economic necessity. I'm refactoring a service right now to offload image generation to the client because the backend GPU costs make the margins impossible otherwise. It makes the architecture much messier, but unless you have VC money to burn on compute, the user's hardware is the only free resource you have.