Comment by int_19h
6 days ago
That doesn't sound realistic to me. What is your breakdown on the hardware and the "top 5 best models" for this calculation?
6 days ago
That doesn't sound realistic to me. What is your breakdown on the hardware and the "top 5 best models" for this calculation?
Look up AMD's Strix Halo mini-PC such as GMKtec's EVO-X2. I got the one with 128GB of unified RAM (~100GB VRAM) last year for 1900€ excl. VAT; it runs like a beast especially for SOTA/near-SOTA MoE models.