Comment by k1w1

This is the part that really stood out to me.

> It’s not only new projects putting strain on the grid though. The report found that an estimated 13 percent of US cloud consumption, totaling more than 3 gigawatts, comes from so-called "zombie" workloads—abandoned test environments and unused applications that continue to draw power without doing any useful work.

Containerized sandbox environments for AI can be incredibly wasteful. If those sandboxes are kept available so the user gets sub-second access that is a continual user of RAM, and thus overall computing resources. We built the first version of https://www.aha.io/builder/overview using a typical containerized environment - just like you see with products like Replit - but were appalled at the inefficiency and waste. We rebuilt it from the ground-up to use shared architecture instead with Javascript-level isolation, and almost zero waste. Using shared computing instead of containers means instant startup time, and (almost) zero resource consumption when not active. You still consume disk to store the artifacts, but there is no ongoing RAM or CPU.

I think a reckoning is coming for container-based AI systems too. We are seeing tokens trend towards reflecting the actual cost, and I think the same will be true of containerized runtime environments too.