Comment by bigyabai

It is fair, it should just be contextualized with a comparison of 13B or 32B models as well. This is one of those Apple marketing moves where a very specific benchmark has been cherry-picked for a "2.2x improvement!" headline that people online misconstrue.