Comment by selcuka
3 months ago
It was a 1.5B parameter model. It was still impressive for 2019, but yeah, it was nothing to worry about.
3 months ago
It was a 1.5B parameter model. It was still impressive for 2019, but yeah, it was nothing to worry about.
1.5B and insanely under-trained. Size didn't mean much before the scaling laws papers came out.