Comment by jononor

I think the comparison with fab nodes is suitable. We do not know how much performance gains are going to come from it, but we do know it is going to be very expensive.

Data availability for LLM is becoming trickier. There are at least two avenues being explored: A) Synthetic data (in controlled ways) and B) Video data, in particular multi-modal embeddings between image/audio/text sequences. This may enable several magnitudes of increase in compute.