Worth a white paper to see which costs less energy, using Apple's built in LLM, or downloading and displaying all the FB ads using radio, playback, and screen animation energy.
Even the data-center-grade models are less intensive than people think. We recently installed a heavy-duty AI server (Gaudi2 cluster with 768GB VRAM) to run larger models up to 400B parameters on, and at full load, its power draw is about equivalent to a consumer stove or clothes dryer.
If you're running thousands of those concentrated into a single data center, your aggregate power draw is going to be huge, but a single server is entirely reasonable for a small business, or even a home user, to operate.
The giant ones in data centers are models that broadly encompass a variety of data for general purpose applications. You can certainly train models specifically for processing DOM content, making them smaller and more efficient.
Apple Intelligence? I mean, you're only processing for the time your "AI|browser|agent" is acting as a firewall between you and Meta. Cache locally after processing and filtering. Use alongside the accessibility API. LLMs can, in many cases, reliably solve CAPTCHAs. I find it difficult to imagine they cannot defeat Meta ad blocking countermeasures.
Not really, no. LLMs get more computationally efficient and hardware gets more power efficient with each passing day. We're already at the point where reasonably competent models can be run a laptop GPU off of battery power. NPUs are even more power efficient, (though a bit slower).
It keeps changing all of the time, so it might be easier to have Facebook load up in a headless browser, that takes a screenshot and sends it to an LLM and uses that to convert into a standard JSON document that can be rendered nicely. Or something like that.
The power cost of doing this widely would be staggering, surely?
Worth a white paper to see which costs less energy, using Apple's built in LLM, or downloading and displaying all the FB ads using radio, playback, and screen animation energy.
Local models are not the same as the giant ones in data centers. They're in about the about the same ballpark as running a AAA game on max settings.
Sure, it's more computationally-expensive than running an HTML selection, but it's also not "staggering" by any reasonable stretch.
I think “staggering” is a pretty accurate way of putting it if blocking ads is in the same ballpark as running a AAA game on max settings.
4 replies →
Even the data-center-grade models are less intensive than people think. We recently installed a heavy-duty AI server (Gaudi2 cluster with 768GB VRAM) to run larger models up to 400B parameters on, and at full load, its power draw is about equivalent to a consumer stove or clothes dryer.
If you're running thousands of those concentrated into a single data center, your aggregate power draw is going to be huge, but a single server is entirely reasonable for a small business, or even a home user, to operate.
The giant ones in data centers are models that broadly encompass a variety of data for general purpose applications. You can certainly train models specifically for processing DOM content, making them smaller and more efficient.
Apple Intelligence? I mean, you're only processing for the time your "AI|browser|agent" is acting as a firewall between you and Meta. Cache locally after processing and filtering. Use alongside the accessibility API. LLMs can, in many cases, reliably solve CAPTCHAs. I find it difficult to imagine they cannot defeat Meta ad blocking countermeasures.
EFF: Adversarial Interoperability - https://www.eff.org/deeplinks/2019/10/adversarial-interopera...
Not really, no. LLMs get more computationally efficient and hardware gets more power efficient with each passing day. We're already at the point where reasonably competent models can be run a laptop GPU off of battery power. NPUs are even more power efficient, (though a bit slower).
Depending on the model, I’m guessing it’s dwarfed by just about any electron app
It keeps changing all of the time, so it might be easier to have Facebook load up in a headless browser, that takes a screenshot and sends it to an LLM and uses that to convert into a standard JSON document that can be rendered nicely. Or something like that.