Comment by ivewonyoung

5 months ago

>For all their flaws, LLMs are so much better

But LLMs get their answers from StackOverflow and similar places being used as the source material. As those start getting outdated because of lack of activity, LLMs won't have the source material to answer questions properly.

11 comments

ivewonyoung

Reply
porcoda  5 months ago

I regularly use Claude and friends where I ask it to use the web to look at specific GitHub repos or documentation to ask about current versions of things. The “LLMs just get their info from stack overflow” trope from the GPT-3 days is long dead - they’re pretty good at getting info that is very up to date by using tools to access the web. In some cases I just upload bits and pieces from a library along with my question if it’s particularly obscure or something home grown, and they do quite well with that too. Yes, they do get it wrong sometimes - just like stack overflow did too.

  • cyral  5 months ago

    The amount of docs that have a “Copy as markdown” or “Copy for AI” button has been noticeably increasing, and really helps the LLM with proper context.

  • inferiorhuman  5 months ago 

       they’re pretty good at getting info that is very up to date by using tools to access the web

    Yeah that's a charitable way to phrase "perform distributed denial of service attacks". Browsing github as a human with their draconian rate limits that came about as a result of AI bots is fucking great.

badthingfactory  5 months ago

StackOverflow answers are outdated. Every time I end up on that site these days, I find myself reading answers from 12 years ago that are no longer relevant.

  • shawn_w  5 months ago

    I see plenty of old answers that are still very relevant. Suppose it depends on what language/tech tags you follow.

  • zahlman  5 months ago

    There have been many times I have seen someone complain on the meta site about answers being old and outdated, and then they give specific examples, and I go check them out and they're actually still perfectly valid.

terminalshort  5 months ago

Now they can read the documentation and code in the repo directly and answer based on that.

  • croes  5 months ago

    SO had answers that you couldn't find in the documentation and were you can't look in the source code.

    If everything would be well documentated SO wouldn't have being as big as it was in the first place.

nutjob2  5 months ago

I think the industry is quickly moving to syntheticly derived knowledge, or custom/systematic knowledge production from humans.