Comment by bradfa
9 hours ago
A small model would be. I think that’s more the point. It’s definitely not SOTA but it’s fast and energy efficient and local.
9 hours ago
A small model would be. I think that’s more the point. It’s definitely not SOTA but it’s fast and energy efficient and local.
> A small model would be [mobile size]
A ~30mm side for the HC1 tech for an 8b model (still unclear the planned HC2)?
Is that analogue or are they baking floating points into the silicon?
It's entirely possible they're using something like block floating point, where most of the hardware is simply fixed point. AMD's NPU does this, for example.
Nope, a small model would be larger than the whole iPhone SoC.