Comment by post-it

13 hours ago

> a Swift package that makes Claude available as a server-side language model in Apple's Foundation Models framework

Ahh I was hoping for the opposite: all of the existing features of Claude Code but somehow running locally on my laptop's neural engine. A pipe dream on an M2 with 8 GB of RAM, but I had a flicker of hope there.

18 comments

post-it

Reply
satvikpendem  12 hours ago

You can use OpenCode or Pi with SSD streaming so it technically will have all the features, just unbearably slow.

FuriouslyAdrift  13 hours ago

I've found most of the frontier coding models require somewhere between 300GB to 1TB to run with full capabilities.

  • godzillabrennus  12 hours ago

    If only we could buy 1TB of unified memory in a Mac for $1k-$2k in total hardware costs. Apple would basically be able to extinguish the entirety of the market cap for Nvidia, OpenAI, Anthropic, and others all at once.

    In 10 years, I hope my MacBook Pro can run today's frontier models and has 1TB of unified Memory.

    • connicpu  11 hours ago

      The Nvidia GB300 DGX Station, which isn't even going to hit 1TB total memory, is expected to launch at almost $100k. Bit of a pipe dream with memory prices where they're at.

      1 reply →

    • manoDev  12 hours ago

      I’m bullish on Apple because of that. Tech waves always oscillate between mainframe/thin-client models at first, then commodity hardware catches up. Apple is well positioned to deliver that with the M series, all it takes is for the current AI bubble to pop a bit and memory costs go down.

    • dboreham  12 hours ago

      The people who train the frontier models want to recover their costs, so they're not going to let you do that.

    • bigyabai  9 hours ago

      > Apple would basically be able to extinguish the entirety of the market cap for Nvidia

      I don't think you understand why people buy Nvidia hardware if you're beating the "just add more dual channel DDR, bro" drum. Apple wouldn't even be able to extinguish AMD with a product like that, it's all slow memory being fed into a raster-first GPU architecture.

  • pstuart  12 hours ago

    The work on LLM in a Flash will probably help, and Apple's NVMe architecture is well suited to maximize throughput could allow their devices to work better on larger models than other vendors.

jubilanti  11 hours ago

> all of the existing features of Claude Code but somehow running locally on my laptop's neural engine

You can use environment variables to have claude code query literally any endpoint you choose as long as it has a compatible API.