Comment by jackmott42
12 hours ago
I a common monument is: "I need to wash my car. The car wash is 100 meters away. Should I walk, or drive?"
The general public has heard of this, tried it on free tier AI, and uses it as an example of how dumb AI is.
Anthropics cheapest tier paid AI, on lowest effort, gets this right.
Sure, they fix these embarrassing things as fast as they get pointed out.
Today I gave a free tier AI a photo of a burning ship posted on Twitter, and asked it to confirm it was an oil tanker. It said no, it was a bulk grain carrier, and gave a specific ID. I pointed out that the bulk grain carrier it had identified was painted black, not red (like most liquid carriers) like the one in the photo (as well as other obvious differences).
That is just one example. The free tier models continue to make very dumb mistakes all the time. Maybe the top secret models, too dangerous for the public, are able to distinguish a black ship from a red one. Who knows.
Using the paid anthrotpic models daily, it has not been my experience that they are just fixing public examples of foolishness, but are generally getting smarter. For example I have my own private, obscure bike racing physics test I use that the newer low tier models pass now.
Before fable, the top tier models didn't pass it. Now even Haiku at low effort does.