Comment by eek2121
3 days ago
I feel like reasoning is a useless metric, and we should instead focus on results, and time to achieved said results. That is a more useful metric overall.
3 days ago
I feel like reasoning is a useless metric, and we should instead focus on results, and time to achieved said results. That is a more useful metric overall.
Reasoning level influences both of those things dramatically. So we need to have it available as a parameter for comparisons.
not a metric, it's a parameter used at inference time
Yes, but what metric defines what constitutes a given reasoning level? That's what's arbitrary here, not the label (which is also arbitrary).