← Back to context Comment by 8note 3 months ago which non-transformer neural networks are matching frontier performance using compute scale? 1 comment 8note Reply paytonjjones 3 months ago BERT is a transformer! The unlock happened within transformers, yes, but they were not exactly super new or innovative architectures at that time. The scale was the main innovation that brought us from BERT to today's frontier.
paytonjjones 3 months ago BERT is a transformer! The unlock happened within transformers, yes, but they were not exactly super new or innovative architectures at that time. The scale was the main innovation that brought us from BERT to today's frontier.
BERT is a transformer! The unlock happened within transformers, yes, but they were not exactly super new or innovative architectures at that time. The scale was the main innovation that brought us from BERT to today's frontier.