Comment by meroes
1 year ago
At a certain level they are identical problems. My strongest piece of evidence is that I get paid as an RLHF'er to find ANY case of error, including "tokenization". You know how many errors an LLM gets in the simplest grid puzzles, with CoT, with specialized models that don't try to "one-shot" problems, with multiple models, etc?
My assumption is that these large companies wouldn't pay hundreds of thousands of RLHF'ers through dozens of third party companies livable wages if tokenization errors were just that.
> hundreds of thousands of RLHF'ers through dozens of third party companies
Out of curiosity, what are these companies? And where do they operate.
I'm always interested in these sorts of "hidden" industries. See also: outsourced Facebook content moderation in Kenya.
Scale AI is a big one who owns companies who do this as well, such as Outlierai.
There are many other AI trainer job companies though. A lot of it is gig work but the pay is more than the vast majority of gig jobs.