Comment by ttoinou

The next mac Ultra will allow to run a big model locally with acceptable speed. But we need people to optimize it for that computer, and we’ll be more limited in models we can choose from

128GB is enough to run a large model, quantized, REAPed, with MoE and fast SSD for model weights