Comment by dr_dshiv
4 years ago
Well, because we want rational language models. Something with a sense of truth.
Math is not irrelevant—and I’m sure it’s a solvable problem with language models.
4 years ago
Well, because we want rational language models. Something with a sense of truth.
Math is not irrelevant—and I’m sure it’s a solvable problem with language models.
But if it's rational and has a sense of truth, then it's AGI. Which I don't think is impossible or even unattainable within a reasonable amount of time, but we're .001% of the way there, not 50% or 75%.
These models are fascinating, but the problem 'a lot of the things this model generates lack any semantic meaning' is inherent and likely insurmountable without connecting the model to other, far more complex models that haven't been built yet.
We are at the level where our models can consistently generate blocks of text with full sentences in them that make grammatical sense. Which is pretty cool.
But the next step is being able to consistently generate full sentences that make grammatical sense and usefully convey information. And while the current models do that a lot of the time, they don't do that all of the time because they don't and can't know the difference without essentially being a different thing. Because to do that consistently, we need an "understanding what things mean" model. Which is many orders of magnitude larger and more difficult than a text generator.