Local is becoming ever increasingly scarce and cost prohibitive. It's kind of bleak out there right now. A minimum bar to entry for decent local AI (something that can run a 27B tier model with some reasonable context) is going to set you back a year or five worth of AI API token costs.
That's true, but assumes that saving money is the only reason to go local (or that someone doesn't already have the hardware to do it – a Mac with 64gb can run most 27B models). By using a local LLM, you can guarantee that your data stays private; that your workflow always works for you and won't randomly change or stop working at the whims of an upstream provider; and that you can evolve your tools or swap in new models whenever you want, without some weird Effective Altruist culter in SF writing a dissertation on the model's supposed mood first.
Local is becoming ever increasingly scarce and cost prohibitive. It's kind of bleak out there right now. A minimum bar to entry for decent local AI (something that can run a 27B tier model with some reasonable context) is going to set you back a year or five worth of AI API token costs.
That's true, but assumes that saving money is the only reason to go local (or that someone doesn't already have the hardware to do it – a Mac with 64gb can run most 27B models). By using a local LLM, you can guarantee that your data stays private; that your workflow always works for you and won't randomly change or stop working at the whims of an upstream provider; and that you can evolve your tools or swap in new models whenever you want, without some weird Effective Altruist culter in SF writing a dissertation on the model's supposed mood first.