Comment by hobofan

4 months ago

Sorry, but what are you talking about? This is a 120B-A6B model, which isn't runnable on any laptop except the most beefed up Macbooks, and then will certainly drain its battery and cook your legs.

5 comments

hobofan

Reply
naasking  4 months ago

You can easily run a quant of this on a DGX Spark though. Seems like a small investment if it meaningful improves Lean productivity.

  • jasonjmcghee  4 months ago

    Is it though?

    Most people I know that use agents for building software and tried to switch to local development, every single time they switch back to Claude/codex.

    It's just not worth it. The models are that much better and continue to get released / improve.

    And it's much cheaper unless you're doing like 24/7 stuff.

    Even on the $200/m plan, that's cheaper than buying a $3k dgx or $5k m4 max with enough ram.

    Not to mention you can no longer use your laptop as a laptop as the power draw drains it - you'd need to host separately and connect

    • naasking  4 months ago

      A single DGX Spark can service a whole department of mathematicians (or programmers), and you can cluster up to 4 of them them to fit very large models like GLM-5 and quants of Kimi K2.5. This is nearing frontier-level model size.

      I understand the value proposition of the frontier cloud models, but we're not as far off from self-hosting as you think, and it's becoming more viable for domain-specific models.

      1 reply →