Comment by ricardobeat

5 days ago

This is meaningless without knowing which model, size, version and if they had access to search tools. Results and reliability vary wildly.

In my case I can’t even remember last time Claude 3.7/4 has given me wrong info as it seems very intent on always doing a web search to verify.

8 comments

ricardobeat

Reply
ay  5 days ago

It was Claude in November 2024, but the “west of equator” is a good enough universal nonsense to illustrate the fundamental issue - just that today it is in much subtler dimensions.

A not-so-subtle example from yesterday: Claude Code claiming to me yesterday assertion Foo was true, right after ingesting the logs with the “assertion Foo: false” in it.

flir  5 days ago

There's something darkly funny about that - I remember when the web wasn't considered reliable either.

There's certainly echoes of that previous furore in this one.

  • fc417fc802  5 days ago

    > I remember when the web wasn't considered reliable either.

    That changed?

    There are certainly reliable resources available via the web but those definitely account for the minority of the content.

    • flir  4 days ago

      I think it got backgrounded. I'm talking about the first big push, early 90s. I remember lots of handwringing from humanities peeps that boiled down to "but just anyone can write a web page!"

      I don't think it changed, I do think people stopped talking about it.

  • SpicyLemonZest  4 days ago

    The web remains unreliable. It's very useful, so good web users have developed a variety of strategies to extract and verify reliable information from the unreliable substrate, much as good AI users can use modern LLMs to perform a variety of tasks. But I also see a lot of bad web users and bad AI users who can't reliably distinguish between "I saw well written text saying X" and "X is true".