Comment by gibsonsmog

Yeah, local is clearly the future. Even beyond the cheap Chinese models you can install the apfel[1] stuff if you're on a mac and want a quick available onboard cli option. And I'm sure people will adapt the Flash-MoE[2] integration to be even better soon as well.

[1] https://apfel.franzai.com/ [2] https://github.com/danveloper/flash-moe