Comment by origini

I try local models regularly on 2x 4090 and struggle with context length constantly. 48GB on GPU and 128GB from RAM.

the original comment mentioned only 16GB of VRAM.

You know talk about Open Router and 27B dense model?

We are not comparing an even more expensive Hardwaresetup to run local models with proper context window and parameters.

Can i run some open model on a super expensive Mac Book or co with some kind of okayisch results? Yeah sure but you know what the t/s will be and yeah Frontier is still Frontier and Kimi, GLM, DeepSeek are not Frontier. They lag half a year.

You are not getting Opus 5 or Sonnet at home for now.

Why are you so offended by this?