Comment by turtlesoup

6 years ago

That's not ideal and probably picking up some of the original training set from GPT-2 (this model is bolted on top of it)! How about DUOLINGOLOGY instead https://bit.ly/3fPGP8q

I'm using a blacklist to reject "real" words but it's surprisingly hard to build for rare words. I'm up to ~600K items after parsing Wikipedia tokens and it still doesn't capture everything.

4 comments

turtlesoup

AdamTReineke 6 years ago

  airpods
  air·pods
  a large pair of wings and wings of a bird or other flying 
  animal, typically used as a guide for a figure skater and 
  paraglider
  "a pair of airpods"

petargyurov 6 years ago

That's understandable! Another one I got was "whitepaper".

In any case, it's all pretty impressive! Defintely something one can use for creative writing or fantasy world building.

kube-system 6 years ago

It seems to make compound words just like humans do

gigafactory

nonplayable

waterboy

pepperjack

unreimbursed

interop

nonalloyed

backdoored