Comment by kmike84
3 hours ago
Yeah, makes sense; e2e is different and valuable, KLD is not a replacement.
As for KLD, have you tried it on something which is even closer to e2e task, like agentic traces from https://huggingface.co/datasets/nvidia/Nemotron-Cascade-2-SF... or similar datasets?
Nope.
If you would like to do so, it is easy (and orders of magnitude cheaper) than running benchmarks.