Comment by HarHarVeryFunny
5 months ago
Efficiency of RL training is a cost issue for the organization training the model, but the capability/utility of the trained model depends on the specifics of the RL training data and training regime (as well as the strength of the base model), which seems to be what differentiates the SOTA coding models.
It's neat that Z.ai are opensourcing slime, and are themsleves using DeepSeek's Sparse Attention - a different approach to that of the big US companies.
Rumor has it that the big proprietary companies are also using DSA now.