Comment by windexh8er
6 days ago
I don't believe everything in my world is as efficient as it could be. But I genuinely think about the costs involved [0]. When doing automations that are perfectly handled by deterministic systems why would I put the outcomes of those in the hands of a non-deterministic one? And at that cost differential?
We know a few things: LLMs are not efficient, LLMs are consuming more water than traditional compute, we know the providers know but they haven't shared any tangible metrics, and the build process involves, also, an exceptional amount of time, wattage and water.
For me it's: if you have access to a supercomputer do you use it to tell you a joke or work on a life saving medicine?
We didn't have these tools 5 years ago. 5 years ago you dealt with said "drudgery". On the other hand you then say it can't do "most things I do". It seems as though the lines of fatalism and paradox are in full force for a lot of the arguments around AI.
I think the real kicker for me this week (and it changes week-over-week, which is at least entertaining) is when Paul Graham told his Twitter feed [1] a "hotshot" programmer is writing 10k LOC that are not "bug-filled crap" in 12 hours. That's 14 LOC per minute. Compared to industry norms of 50-150 LOC per 8 hour day. Apparently,this "hot-shot" is not "naive", though, implying that it's most definitely legit.
[0] https://www.sciencenews.org/article/ai-energy-carbon-emissio... [1] https://x.com/paulg/status/1953289830982664236
> When doing automations that are perfectly handled by deterministic systems why would I put the outcomes of those in the hands of a non-deterministic one?
The stuff I'm punting isn't stuff I can automate. It's stuff like, "build me a quick command line tool to model passes from this set of possible orbits" or "convert this bulleted list to a course articulation in the format preferred by the University of California" or "Tell me the 5 worst sentences in this draft and give me proposed fixes."
Human assistants that I would punt this stuff to also consume a lot of wattage and power. ;)
> We didn't have these tools 5 years ago. 5 years ago you dealt with said "drudgery". On the other hand you then say it can't do "most things I do".
I'm not sure why you think this is paradoxical.
I probably eliminate 20-30% of tasks at this point with AI. Honestly, it probably does these tasks better than I would (not better than I could, but you can't give maximum effort on everything). As a result, I get 30-40% more done, and a bigger proportion of it is higher value work.
And, AI sometimes helps me with stuff that I -can't- do, like making a good illustration of something. It doesn't surpass top humans at this stuff, but it surpasses me and probably even where I can get to with reasonable effort.
It is absolutely impossible that human assistants being given those tasks would use even remotely within the same order of magnitude the power that LLM’s use.
I am not an anti-LLM’er here but having models that are this power hungry and this generalisable makes no sense economically in the long term. Why would the model that you use to build a command tool have to be able to produce poetry? You’re paying a premium for seldom used flexibility.
Either the power drain will have to come down, prices at the consumer margin significantly up or the whole thing comes crashing down like a house of cards.
> It is absolutely impossible that human assistants being given those tasks would use even remotely within the same order of magnitude the power that LLM’s use.
A human eats 2000 kilocalories of food per day.
Thus, sitting around for an hour to do a task takes 350kJ of food energy. Depending on what people eat, it's 350kJ to 7000kJ of fossil fuel energy in to get that much food energy. In the West, we eat a lot of meat, so expect the high end of this range.
The low end-- 350kJ-- is enough to answer 100-200 ChatGPT requests. It's generous, too, because humans also have an amortized share of sleep and non-working time, other energy inputs/uses to keep them alive, eat fancier food, use energy for recreation, drive to work, etc.
Shoot, just lighting their part of the room they sit in is probably 90kJ.
> I am not an anti-LLM’er here but having models that are this power hungry and this generalisable makes no sense economically in the long term. Why would the model that you use to build a command tool have to be able to produce poetry? You’re paying a premium for seldom used flexibility.
Modern Mixture-of-Experts (MoE) models don't activate the parameters/do the math related to poetry, but just light up a portion of the model that the router expects to be most useful.
Of course, we've found that broader training for LLMs increases their usefulness even on loosely related tasks.
> Either the power drain will have to come down, prices at the consumer margin significantly up
I think we all expect some mixture of these: LLM usefulness goes up, LLM cost goes up, LLM efficiency goes up.
3 replies →