Comment by ks2048
3 days ago
So, Meta will spend millions/billions on building open-weight models and then people will send Apple $20,000 to run them. Seems like a good deal for Apple.
3 days ago
So, Meta will spend millions/billions on building open-weight models and then people will send Apple $20,000 to run them. Seems like a good deal for Apple.
Exactly why does Apple need to do any model training/development? They are out here selling the shovels.
Apple’s Photos ecosystem is privacy oriented and as a result some (now) baseline features require the use of local models.
The Foundation models keep a surprisingly decent set of data available for fast filtering by a local LLM.
I’ve found a slower, second pass, is needed by Qwen to clean it up. (I am running on the new Mac Mini m5 pro 64gb.)
I’ve run my own tests using a date range, a short description of a red cedar planter box build I did last year, and a set of known correct photos.
There is some cool stuff you can do without sending your family’s pics up to any of the frontier model providers.
I've found some strange guardrails with the Foundation models. Like blocking describing of any photo with a single woman in it. It does this relatively consistently but with a different subset every time. Like out of 10 photos in the set, it's skips 6 then the next run, it skips 3. Have you seen guardrail issues like this?
I switched to Gemma 4 instead.
2 replies →
No. Apple’s Photos ecosystem uploads all Photos effectively unencrypted to Apple in a way that is readable by Apple, allowing Apple to do server-side analysis of all content. This content is also provided to US or Chinese federal authorities without due process or warrants.
Nothing about iCloud Photos is private.
5 replies →
I don't want any of these features and yet they've been forced upon me and are now getting in the way of me actually using my computer how I want. Games and heavy applications slow down and stutter because of "AI" background processes, even with App Tamer, my machine runs too hot to even touch when just idling. Compared to when I got my M2 in '23 and thought, "This is the fastest and snappiest machine I've ever used", my experience with an M5 and the latest macOS version is nothing short of an anti-consumer hellscape.
35 replies →
Exactly my thought as I read it.
They make and sell the hardware that will actually make AI something everyone can used without needing to rely on OpenAI/Anthropic et al.
Apple has never had a moat, anyone else could have built this sort of hardware but they haven't.
So they don't care which model wins, because it will always move out to the edge of the network, and they own a large chunk of that edge.
> Apple has never had a moat, anyone else could have built this sort of hardware but they haven't.
I think both points are very incorrect. Apple has a massive moat. And Apple is making hardware (specifically, CPUs) that are two years ahead.
Well, the moat is that cpu/npu development is very hard, requires talented engineers, and requires access to cutting edge fab process, all of which in turn costs a lot of $$$. So, sure, “anyone” who has all that can definitely breach Apple’s moat.
4 replies →
Ermmm what?
Apples largest competitive advantage has always been and always will be - trust combined with epic marketing to make the vast majority of people aware of it.
Trust of quality, privacy w.r.t data etc.
I’m sorry but I’ve seen so many bad takes re. Apple over the years. Many of you think you are so smart on topics outside of your domain - but you’re not.
5 replies →
Nvidia is also selling the shovle but they train their own LLM and by doing so they can fine tune their CUDA/Chip to have a better shovel than others.
Nvidia's own model does not have to beat OpenAI or any SOTA model. Nvidia just need to build up the in house expert and knowledge to make a better chip since hyperscalers and none hyperscaler like Meta is building their own AI chip now.
Apple can chose not to train their own model to learn how to sell a better shovel.
> They are out here selling the shovels
just wish their containerization / linux compat wasn't so nerfed. Just moved my homelab's inference engine over to a GB10 from a Mac MTP setup to have first class k8s support, as well as open up access to the broader CUDA ecosystem (cuOpt etc.).
They’re not really. They make the things useful for two classes of users: Mac Studio for people running models locally and Siri backed by Gemini for voice interaction.
They’ll likely embed text models more deeply in the future, but the history of “Apple Intelligence” doesn’t give a lot of hope that they’ll nail it.
So they’re not selling shovels (a precursor to making the thing) but products and services to use it.
apple is pretty well set up for this because they do all their hardware in house. if they wanted to, they could pivot in this direction. Ternus is also a hardware guy.
That's just BS cope. They tried and miserably failed - and still are failing badly. Remember, they were the ones to push for AI with "Siri". Many firings later and 2 CEOs later we are still exactly in square one. The fucking thing can't even get who I want to call, let alone set an alarm without touching my phone FFS.
What is there to cope about? Apple is the last vertical computer company left from the 1980s. Apple Silicon just gets better every year. They have also managed to replace several companies that well gave them problems over the years or said no to doing anything that involved moving into the future. i.e, Motorola of Schaumburg, Illinois, IBM and Intel one company’s out of business. and the other two are trying to hang on to past glories.
By the way, the new Apple folding phone appears to be a hit. a real hit the last test left is whether or not it is as good as it appeared to have been in the demonstrations. only time will tell when it gets into the hands of the general public.
2 replies →
I have no interest in Siri or always-on AI assistants. I turn as much of that off as I can.
I do have an interest in local, private inference though and I am willing to pay for that.
Apple segmenting the market is actually a good strategy. Horses for courses, as they say.
1 reply →
Apple playing 3d chess here -- they can just absolutely kill it in making the HW available for everyone to run models at home, and will do a better job than Nvidia especially when M7 comes out which is specifically designed for AI workloads.
Both companies are profitable and AI is not their main line of business. This affects pure AI companies.
As open weight models continue to match frontier models, the race to $0 will continue to benefit supporters of running open weight models locally.
How big is the local LLM market? I have trouble imagining Apple will move enough units to make $20k/ea worth it for them as a business segment.
100,000 units a year would be $2 billion annually.
And usually, most of a company's revenue comes from the heaviest spenders.
So figure there are some out there who will by tens of these because they have some "idea to change the world". (Whether true or not.) Some startups, etc.
I'd wager, across planet earth right now, it should be trivial to push 100,000 units per year. Probably closer to 500,000 units per year. So that's an easy $10 billion a year.
$2-10 billion is meaningful for any company. Even one as large is Apple.
Especially if it's a first step towards a stronger foothold in the AI inference space.
They can then expand down market to grow the pie further.
For example, with truly dedicated inference boxes.
And given the demand for $7,000 Sparks and sold out $5,000 GPUs, the market for these $20,000 machines is likely much higher than my 100,000 unit lower bound. (Earth is a big place. It doesn't take much excitement to generate massive sales in a space with currently insatiable demand.)
The only question you need to ask is whether or not Apple will do anything to design and engineer memory in-house, the money, and the capability is there that is the challenge. because you don’t need to select $20,000 computer if you get that under control by bringing it in-house.
In short, Apple needs to roll up there sleeves and get busy the three hyenas of memory and that nameless memory company in China will not sell you memory when you need it at a reasonable price. In the past Motorola said no, IBM said no, and Intel said no. This is another one of those times for Apple to adapt by designing and engineering around the competition.
The memory constraint isn’t design. It’s fab capacity. Apple has exactly 0 fabs. By the time they hired domain experts and built fabs and tuned to the point of reasonable yields, the crunch would have been over for 5 years, and apple would have wasted 20 billion with no return.
They have been sold out of a lot of units, due to people developing AI, so apparently, it is significant.
I think the bet is that it will get bigger, both at the consumer level, and with orgs eventually discovering ready-made solutions for having their own on-prem inference.
Probably worth it just as an investment into their developer ecosystem. They could probably sell at a deep loss and still justify it.
They have the best strategy of bigtechco: let others figure out the squishier layers while they focus on selling the best hardware. Privacy has incredible value to a lot of individuals and industries. $20-80k in hardware that *appreciates* the moment you buy it is a reasonable investment for a lot of SMBs, and still less than a single Nvidia rig. Privacy pays.
I disagree.
I think the iOS App Store is going to be in steep decline in a few years. People will generally have a few apps: ChatGPT, Instagram, messaging, and something like a Photos/Camera app.
Everything else will be done in your AI app.
I've been hooking up more and more data and plugins into ChatGPT/Claude. I only want to use it. I don't want to use other apps if I don't have to. For example, research, buy, plan. I prefer not to use other apps.
I think all it takes is for OpenAI to make a phone from the ground up for this and Apple is going to have serious competition.
If we're being frank here, Apple's AI strategy lags behind Nvidia, AMD and even Google. Nvidia has CUDA, AMD ships CDNA, Google has the TPU family, and Apple... well, Apple discontinued the rackmount Mac Pro, forfeit the HPC ARM datacenter to Nvidia's Grace CPUs, refuses to sign CUDA drivers, and spent half a decade designing raster GPUs instead of GPGPUs.
All of their competition has real inference and training stacks. Apple ships an iGPU and a smartphone NPU, they're reliant on Nvidia and Google hardware to deliver Apple Intelligence.
Saying they are lagging assumes they are in the same race
They aren’t
Apple is a high net worth consumer technology integrator that focuses on highest coherence user experience. They pick the most efficient stable technology, integrate it and sell the package
Apple only needs to lead in precisely that they don’t need to lead in literally anything else
in fact if they tried to lead in those other things they would give up their integration capacity
9 replies →
How many 5080 16 Gig cards would you have to buy to run the same model, and what would that cost?
idk but naive me sees the article say the model needed 158GB of RAM so I guess 10x 16GB GPUs? Apparently retail price is $1k per 5080 16GB but they're actually selling at $1.5k so I guess $15k for just the GPUs.
158GB number is for Mistral Medium 3.5 128B model, Muse Spark is way larger.
3 replies →
Now scale that up to Nvidia and you've understood the entire AI market
Still cheaper than the equivalent Nvidia GPU setup btw.
The NVIDIA GPU can be changed as new models become available, without throwing away the whole computer.
And the Mac Studio can continue to be used as a regular computer. (Hilariously over-qualified, I suppose. But, no, I got a maxed out, 128 GB Mac Mini earlier in the year—when you still could—and I find it a snappy daily driver in my 'man cave' downstairs.)
3 replies →
The lifespan of those AI GPUs is like 5 years, I have macs from 2008 that still turn on and function.
1 reply →
I wonder how many units Meta bought to convince Apple to do a collaborative advertisement.
That is generally how hardware gets sold yes
[flagged]
It won't be free for Apple to use Meta's model in the future.
Google, who is far more qualified, and is getting a measly 1 billion dollar per year refund on that $20 billion dollars they currently paying Apple for a default search position on Mac OS. Meta, whose software prowess is questionable, when compared to Google or Microsoft isn’t even in the game they are getting how should I put this Apples charity/tolerance which is all OpenAI and Google are currently getting.
One is getting a $1 billion refund from what they already pay Apple, and the other two are getting nothing from Apple.
Meta won’t be getting anything from Apple in the future, other than tolerance/charity.
Even in the unlikely case that does happen, there are plenty of other open weight models like Qwen, GLM, and Mistral to name a few