Comment by bigyabai
4 hours ago
You can layer models across multiple GPUs for inference. If you pooled it into one system, that would be 72gb of some pretty damn fast memory. Would be great for dense models in the 30-50b range.
4 hours ago
You can layer models across multiple GPUs for inference. If you pooled it into one system, that would be 72gb of some pretty damn fast memory. Would be great for dense models in the 30-50b range.
No comments yet
Contribute on Hacker News ↗