Comment by mjburgess
4 years ago
> 500 bn parameter model
That solution has a compressed representation of half the internet.
NNs are "garabled nonesense" insofar as they try to generalise; insofar as they are search engines, they provide apparent sense by just repeating something in their database (= weights).
Google, likewise, returns joke explanations.
"The Internet" is much, much bigger than 500bn floats. The latest snapshot of archive.org has 505 billion web pages itself.