Comment by aberzun

It will win - in the sense that AI too will become a freely available resource. You can't stop progress.

My bet is that once cost-efficiency becomes a priority, we will figure out ways to get away from the expensive GPU infrastructure on figure out how to architect models for CPUs. I still remember that Microsoft paper about ternary weights.