Comment by adamsb6

I realized that should I end up getting laid off soon I won't have an unlimited token budget and for the workflows I've settled into it would be quite expensive. So I was exploring what it would take to run open models at home.

Was quite disappointed to see that the PC side hasn't kept up. The unified architecture on Macs makes it very hard to justify spending money on a Linux machine for inference workloads.