Comment by lijogdfljk
4 years ago
When can we run something like ChatGPT locally i wonder? Ie like StableDiffusion.
I'm kinda dying for that, honestly. I can't even imagine all the neat applications i'd make of ChatGPT if it was purely local.. but it would take all of my free time to play with it. It's so damn impressive.
Probably not realistic for now to run it locally, GPT-3 has like 175 billion parameters, you need to count around at least 2 bytes in optimistic scenario per parameter so you have around 350 GB of GPU memory, you probably need at least around 15 GPUs with minimum 32 GB of memory each.
Isn’t there an abundance of GPUs from crypto farmers? ;)
I asked it what hardware would be necessary to do that and it said an nVidia V100 GPU lol
I think you can run BLOOM locally, but it's not quite as powerful as this iteration of chatGPT. Also the vRAM requirements are pretty high if you want to run the biggest model.
https://huggingface.co/bigscience/bloom
Agreed. The things I want to do with this don't make sense as a web service.