Comment by piker
2 hours ago
How is a GPT implementation “extremely out of distribution” but “planet scale infra” isn’t? That’s got me totally confused about your point that I was taking seriously.
2 hours ago
How is a GPT implementation “extremely out of distribution” but “planet scale infra” isn’t? That’s got me totally confused about your point that I was taking seriously.
A lot of stuff that underpins planet-scale infra is open source and has been for a long time. The general approaches and architectures for it have also been widely discussed and litigated in the commons so it's not just the code - the specific why is also very well documented.
So despite its importance much of it is actually pretty in-distribution.
Okay, but then that does not square with calling a GPT implementation out-of-distribution, let alone "extremely" so.