← Back to context

Comment by famouswaffles

3 years ago

>Even if they can be tickled into producing a new AI from scratch by the nature of an LLM it's going to be sort of the average of its training set

There's this idea that LLMs somehow end up as some sort of average of training data and it's incredibly wrong.

LLMs learn to make predictions for all states at any time. There is no average they fall into. GPT-4 is not some average of its training data. A "perfect" LLM will predict Einstein as easily as it predicts the dumbass across the street.

6 comments

famouswaffles

Reply
jerf  3 years ago

Which is why I disclaimed it. I hate it when people quote things, cut off the quote, then bitch about the part they cut off.

No the models will not "predict Einstein". They'll predict the most popular interpretation of him at best, and while they is also a simplification, ChatGPT is not sitting on top of the solution to the Grand Unified Theory. It may give a good overview of the consensus, but it will not be able to tell you the correct solution to the problem right now... though it won't be hard to convince it to swear up and down that it has.

  • jameshart  3 years ago

    For ‘predict Einstein’ read ‘predict what an Einstein-level intellect would think (about a given subject)’, not literally replicate what Einstein did.

    You’re focused on the idea of LLMs as collators of ‘things that have been said’, but that’s not all that they collate from their training set.

    • jerf  3 years ago

      Yes, I'm aware of that too. Maybe I just don't feel like spewing a complete description of LLMs into my every post about them.

      To the extent that they may come up with novel ideas, they have no ability to compare them against the true state of the world. This is not exactly a limitation of them per se that could be overcome with more computation, so much as just a structural fact about them; they have no loop where they can form a hypothesis, test it, and adjust based on data. It simply doesn't exist.

      Which is part of why I keep saying that while I'm less impressed than everyone else is with LLMs, the future AIs that will incorporate them but not simply be an LLM is going to really knock people's socks off. Pretty much all the things people trying to convince LLMs to do that they really can't do are going to work in that generation. I have no idea if that generation is six months or six years away but I wouldn't bet much more than a few years.