Comment by bryanlarsen
9 hours ago
SemiAnalysis estimates their profit margin to be 70%. To be losing money on inference implies that their costs are almost 4X higher than SemiAnalysis has calculated. That's not credible.
9 hours ago
SemiAnalysis estimates their profit margin to be 70%. To be losing money on inference implies that their costs are almost 4X higher than SemiAnalysis has calculated. That's not credible.
I don’t see how they could credibly estimate inference costs without knowing the model size.
But we do have a reasonable estimate of model size.