Comment by blunte
7 years ago
Since it seems the general population is becoming less and less able to read stories or information consisting of more than one thought or layer of abstraction, grammatically reasonable strings of words like these may satisfy plenty of people. Indeed, modern television and televised "news" seems to be no more impressive than this.
But to me, this makes very little sense. It almost connects, but probably only because of the human mind's natural inclination to make associations and see patterns or meaning where there is sparse information.
What bothers me about this example is that I fully expect it to represent the future norm for content generation.
It doesn't make much sense. But compare it to the past state of the art – starting with Markov chains, proceeding to things like [1]. They were only able to maintain coherence for a handful of words. Full sentences were almost always nonsensical; you might be able to make sense out of them, but only with heavy application of "the human mind's natural inclination to make associations". Now compare that to GPT-2, which generates not only sentences but entire paragraphs that are most often fully self-coherent; most of the examples in this thread only start to break down across multiple paragraphs.
That's still not enough, and GPT-2 in particular has been heavily overhyped, with fanciful claims that people might use it to generate fake news. (What would the point of that be? You don't need N fake news articles to reach N people, only one, which can be written by a human.)
But it's progress. GPT-2 still feels like science fiction to me. What if a future text generator, maybe even one or two decades down the line, surpasses GPT-2 to the same extent that GPT-2 surpasses the earlier attempts I mentioned? What if that system extends the length of coherence to reliably cover entire articles and essays? What if it gets better at synthesizing the information about the world represented by its training data, so that its output on nonfiction prompts is factually true, rather than mere plausible-sounding nonsense? Is it possible? Perhaps not. But it seems a lot more likely to me now than it did before GPT-2 was created.
[1] http://karpathy.github.io/2015/05/21/rnn-effectiveness/
I agree with you that this doesn't reach near the level of good human authors. There's no long term plot or deep human themes in this. I don't think this will ever replace quality human writing, but it may be able to augment it in cool ways. I personally would love if rather than every guard in Skyrim telling the exact same story, if each guard could have their own stories or comments generated based on things about their life. Human authors could provide high level details and let AI generators fill in the smaller details.
I'm not bashing your effort. From the posts I've seen showing the content, it is quite impressive. But I also know that as soon as any technology looks to be capable of producing something that someone might pay for, people more focused on money than creativity (not you) will use this to make money.
It's a bit like modern pop music. We are well beyond the point where software can not only write the music, but even take the once human voice and recreate it to sing the song. Of course it's not great, if one were to really listen - but it is passable, and it makes money. So it drowns out the remaining bits of real human creativity.
Can you give any examples of such a song?
2 replies →
Chatbots that generate incoherent ramblings that make you feel slightly attached have existed for decades. There is nothing special about these. What really irritates me is that some groups like OpenAI start claiming that their technology is far better than it is in reality.
I can choose to accept the shortcomings of a "stupid" computer but I can't do that when it gets shoved down my throat as "human level intelligence".