Comment by make3

3 months ago

the thing could barely make full grammatical sentences, it's funny to see that even then they were overclaiming the fuck out of their myself

6 comments

make3

Reply
selcuka  3 months ago

It was a 1.5B parameter model. It was still impressive for 2019, but yeah, it was nothing to worry about.

  • make3  3 months ago

    1.5B and insanely under-trained. Size didn't mean much before the scaling laws papers came out.

anuramat  3 months ago

imho it was more reasonable back then to claim "agi soon" -- back when nobody really knew how it scales

  • phire  3 months ago

    They weren't claiming it was dangerous because "AGI soon", that didn't come until later.

    OpenAI were claiming GPT-2 was too dangerous because it could be used to flood the internet with fake content (mostly SEO spam).

    And they were somewhat right. GPT-2 was very hard to prompt, but with a bit of effort it could spit out endless pages that were good enough to fool a search engine, and even a human at a first glance (you were often several paragraphs in before you realised it was complete nonsense.