Comment by ychen306
10 days ago
It's orders of magnitude cheaper to serve requests with conventional methods than directly with LLM. My back-of-envelope calculation says, optimistically, it takes more than 100 GFLOPs to generate 10 tokens using a 7 billion parameter LLM. There are better ways to use electricity.
I work in enterprise IT and sometimes wonder if we should add the equivalent energy calculations of human effort - both productive and unproductive - that underlies these "output/cost" comparisons.
I realize it sounds inhuman, but so is working in enterprise IT! :)
I agree wholeheartedly. It irks me when people critique automation because it uses large amounts of resources. Running a machine or a computer almost always uses far less resources than a human would to do the same task, so long as you consider the entire resource consumptions.
Growing the food that a human eats, running the air conditioning for their home, powering their lights, fueling their car, charging their phone, and all the many many things necessary to keep a human alive and productive in the 21st century are a larger resource cost than almost any machine/system that performs the same work. From an efficiency perspective, automation is almost always the answer. The actual debate comes from the ethical perspective (the innate value of human life).
I suspect you may be either underestimating how efficient our brains are at computing or severely underestimating how much energy these AI models take to train and run.
Even including our system of comfort like refrigerated blueberries in January and AC cooling a 40° C heat down to 25° C (but excluding car commutes, because please work from home or take public transit) the human is still far far more energy efficient in e.g. playing go then alpha-go. With LLMs this isn’t even close (and we can probably factor in that stupid car commute, because LLMs are just that inefficient).
8 replies →
This is a bad argument. Even if a machine replaced my job, I'm still going to eat, run the aircon, charge my phone etc. and maybe do another job. So the energy used to do the job decreased, but the total energy usage is higher because I'm still using the same amount of energy, but now the machine is also using some amount energy that wasn't being used before.
Efficiencies lead to less resources being used if your demand is constant, but if demand is elastic, it often leads to the total resource consumption increasing.
See also: Jevons Paradox (https://en.wikipedia.org/wiki/Jevons_paradox).
Not ALL automation can be more efficient.
Just ask Elon about his efforts to fully automate Tesla production.
Same as A.I. Current LLM-based A.I.s are not at all as efficient as a human brain.
Only slightly joking, but someone needs to put environmental caps on software updates. Just imagine how much energy it takes for each and every discord user to download and install a 100MB update... three times a week.
Multiply that by dozens or hundreds of self-updating programs on a typical machine. Absolutely insane amounts of resources.
Goodhart’s Law will mess all that up for you.
Try to convince the investors. The way the industry is headed is not necessarily related to what is most optimal. That might be the future whether we like it or not. Losing billions seems to be the trend.
Eventually the utility will be correctly priced. It's just a matter of time.
No, it will not be correctly priced. It will reach some kind of local optimum not taking any externalities into account.
We are all dead in a matter of time.
Debt, just like gravity, tends to bring things crashing down, sooner or later.
Sure, but we can start with an LLM to build V1 (or at least a demo) faster for certain problem domains. Then apply traditional coding techniques as an efficiency optimization later after establishing product-market fit.