Comment by nijave
5 days ago
> Me: What was the sushi place near latitude 41 in Columbus? Did it go out of business I don't see it on Google maps anymore
> Sonnet 5 (medium): None of these past chats mention a sushi place — I don't have anything on record about that. Do you remember the name, or roughly which part of Columbus (neighborhood/street) it was near? That'll help me search and check its current status.
Not impressed. It got the name right on high effort one shot but hallucinated the date relativity (Jan 2026 is not last month...). Worked okay on extra. Sonnet 4.6 worked fine on medium, high, and extra one shot.
Edit: as the replies point out, the prompt is definitely ambiguous however Sonnet 5 didn't even extract the semantic meaning "looking for <place> near <compound place>" which all the human replies seem to understand. Even Haiku 4.5 identifies the semantic meaning although it fails to retrieve the correct results
> Haiku 4.5 (reasoning off) I'll help you find information about that sushi place. Let me search for sushi restaurants that were near latitude 41 in Columbus and check their current status.
>_Searched the web_
> I see that "Latitude 41" is a restaurant in Columbus, but it's actually a modern American restaurant, not specifically a sushi place. However, based on your mention of latitude 41, that's what came up. Let me search more specifically for sushi restaurants that may have closed in Columbus around that area.
What was your expectation? That your prompt would trigger a web search, first, before the introspection of past conversations and a training set recall?
How did Sonnet 4.6 respond that was objectively better for your use case?
[flagged]
Try it 25 more times and let us know how it averages out. It's non-deterministic, remember?
I tried 3 more times. Two were nearly identical and 1 recognized Latitude 41 as a restaurant but had a similar useless reply
"Latitude 41" is a business name?
If you gave me this prompt, I'd say "Which Columbus? None of the ones I know about are at a latitude of 41 degrees north or south?"
2 replies →