Comment by janalsncm

GPT2 and 3 work via autoregression. In chat bots, decoders work via autoregression.

> people spend years just post-training BERTs in various ways

Yes, I was one of them. That’s not called “post-training” it’s called fine-tuning.