they are building a product and said the unit economics must make sense, local models have slower latency, unless you run a gpu on for hours which gets expensive fast
Local models will make a lot more sense once we have the scale for it, but when your user count is still small paying cents per image is a much better deal than paying for a GPU either in a data center or physically.
Local models are definitely something I want to dive into more, if only out of personal interest.
they are building a product and said the unit economics must make sense, local models have slower latency, unless you run a gpu on for hours which gets expensive fast
Local models will make a lot more sense once we have the scale for it, but when your user count is still small paying cents per image is a much better deal than paying for a GPU either in a data center or physically.
Local models are definitely something I want to dive into more, if only out of personal interest.