Comment by phire
3 months ago
Yes, Deepseek introduced this optimisation of a common base "expert" that's always loaded. Llama 4 uses it too.
3 months ago
Yes, Deepseek introduced this optimisation of a common base "expert" that's always loaded. Llama 4 uses it too.
I had a sneaking suspicion that I wouldn't be the first to think of it.