Comment by IncreasePosts
3 days ago
You don't necessarily, but each token costs money for the AI to spit out. And probably more money when that output is used as input later. Delegating to a library makes sense financially.
3 days ago
You don't necessarily, but each token costs money for the AI to spit out. And probably more money when that output is used as input later. Delegating to a library makes sense financially.
With local inference on pretty decent local models we have nowadays (Qwen-3.5 and better) it's not much of a concern anymore.
Sure it is - there's still an opportunity cost of spending tokens(time/energy) creating a library from scratch vs using a preexisting well understood API.
Sure, if you've got a £5k laptop
what percentage of people is using local models for anything serious? I reckon single digits if even that. And for a corporate work environment, probably close to 0.