Comment by aswegs8
2 days ago
Kind of peculiar and memorable story for me.
I was on the couch on my Nintendo Switch, playing around with ChatGPT 3 and asked it where to find a specific item in Zelda Breath of the Wild. When it provided a coherent answer I was just dumbfounded. To be fair, the answer was semi-hallucinated but partly true. But it made me realize what kind of breakthrough it must be for some program to provide an answer to this without searching external sources (which it couldn't do yet). Such a small data point, like a drop in the vast sea of human knowledge space.
Prompted me to do some back on the envelope calculation. The weights of this model were a few hundred GBs. I just realized what kind of quantum leap it was to compress this seemingly infinite knowledge space into a few GB of weights.
How is that result any different than what you would have done without AI, which is likely a search for the item name which would have given you a wiki about it and videos on how to get it? Why were you amazed at what, on any other day, is a common query? Really just that it was offline?
Like endymi0n said, for me it was just about the amazement of how qualitatively different this technology was than anything we had before. Not in a functional sense of how it provides an output to anything. That was probably when I used it to spin up an app via Terraform in days despite having not much prior knowledge of infrastructure. Digging through Stackoverflow would have took at least 10 times more time.
not parent, but I kinda have the same thoughts often. Maybe I can’t do inference on them in the same form factor (yet!), but just the fact that the weights of a model that comes close to capturing a close enough approximation of the combined knowledge, experience and intelligence of mankind fits onto a MicroSD never fails to amaze me.
Ok but you can download the entirety of Wikipedia too, in a queryable format..