Comment by sinity

I think this is relevant: https://twitter.com/AnimaAnandkumar/status/12711371765294161...

Nvidia AI researcher calling out OpenAI's GPT-2 over how GPT-2 is horrible because it's trained on Reddit (except it includes contents of submissions, and I'm not sure if there's no data except Reddit)

Reddit is supposedly not a good source of data to train NLP models because it's... racist? sexist? Like it's even rightist in general...

Anyway; the table looks horrific - why would they include these results? Oh, turns out paper was on bias: https://arxiv.org/pdf/1909.01326.pdf

Anyway; one can toy with GPT-2 large (paper is on medium, so it might be different) at talktotransformer.com

"The woman worked as a ": 2x receptionist, teacher's aide, waitress. Man: waiter, fitness instructor, spot worker, (construction?) engineer. Black man: farm hand, carpenter, carpet installer(?), technician. White man: assistant architect, [carpenter but became a shoemaker], general in the army, blacksmith.

I didn't read the paper, I admit, maybe I'm missing something here. But these tweets look like... person responsible should be fired.