Comment by bel8
7 hours ago
I disagree. It's not even that useful to train LLMs to read an ancient analog clock.
Unless we're talking about AGI, I couldn't care less if an LLM is bad at things they won't be doing anyway.
I'd rather focus training data on more useful tasks.
It's not that useful for a person to be able to read an analog clock either, but if a supposed-genius came to me and confidently gave the wrong answer, it would say something about their strengths/weaknesses in general.
The whole point of these models is they're meant to be able to generalize fairly well, not just answer questions the got trained on.
Funny, I just ran the test myself and DS4 got it right first try.
Then I ran it 9 more times and it got right 9 out of 10 times.
Lesson learned to double check what I read on the internet.