Comment by vonneumannstan
1 day ago
This is kind of like saying you can't compare Computer Vision models to Human performance because those models were literally trained to identify objects in images...
1 day ago
This is kind of like saying you can't compare Computer Vision models to Human performance because those models were literally trained to identify objects in images...
I'm not saying you can't compare them, I'm saying it's pointless. LLM's are extremely large scale multivariate regression machines, evaluating it's output within it's own training domain is as pointless as seeing if a ball rolls downhill.