Comment by paxys

17 hours ago

Local setups aren't going to make sense purely from a cost perspective, and definitely not when you are buying Apple hardware. AI subscriptions are too highly subsidized right now.

15 comments

paxys

Reply
monatron  17 hours ago

I think your last point is exactly why I'm so interested in local models. The current landscape doesn't feel sustainable. The last few months we've seen the big providers (OpenAI, Anthropic) start to play with usage limits, resets, banked resets, pulling models, etc. I think local models are close to the point where, with a sufficiently well-architected harness, you can get results that are on par with the experience you'd have with cloud inference. It is nice to know that I have hardware under my desk that I control with open weight models that I can interact with on my terms.

  • polnoner  9 hours ago

    It is certainly not sustainable but that is exactly why I want to use $400-$800 of resources a month for $20 while the deal last.

    It is really a once in a lifetime deal.

    Once the deal is over the local models will be better than what I am using now anyway and the hardware will be all the better than what I can get now for the price.

  • htrp  17 hours ago

    yes, you have to unfortunately adopt a 2025 mindset when working with smaller local models because that's where they are compared to the frontier.

  • julianlam  17 hours ago

    > local models are close to the point where, with a sufficiently well-architected harness, you can get results that are on par with the experience you'd have with cloud inference

    In my opinion, 98% of the work most devs would send to an AI can be capably achieved with a local model and a frontier-level model is overkill.

    The goalpost moving feeds right into Anthropic and OpenAI's interests.

apparent  13 hours ago

> AI subscriptions are too highly subsidized right now

I've been running into annoying limits with Claude recently. It gives me like 5 questions over the course of 15 mins and then tells me to wait 5 hours. When companies can change things up to make the base subscription nearly useless (the last question always gets messed up, too), then you realize the value of owning your own infrastructure.

  • adastra22  12 hours ago

    On the $200/mo plan I have never hit a five hour limit, and I struggle to use my full credits each week.

    $200/month is vastly cheaper than owning and operating comparable hardware.

    • apparent  12 hours ago

      You're right that $200/mo is much cheaper than comparable infrastructure. OTOH, you don't get to have a computer that can also be used for other applications, or which works when the internet is down. Also, you remain tethered to whatever pricing the AI companies want to charge. If AI pricing goes like Uber/Lyft did when the VC cash ran out, then we'll be paying much more in a few years' time. We could look back and think "I wish I'd bought my own setup back in 2026" if it's going to be inevitable.

      And all this before you get into privacy/security/compliance stuff.

      1 reply →

    • 0x457  12 hours ago

      > On the $200/mo plan I have never hit a five hour limit, and I struggle to use my full credits each week.

      I started tasking fable with huge projects over the weekend and now I hit at very least fable limit by monday.

      1 reply →