← Back to context Comment by adgjlsfhk1 6 hours ago The size of model we're talking about running doesn't need much if any dram. 1 comment adgjlsfhk1 Reply trollbridge 34 minutes ago The chatjimmy demo is using a model that needs 6-18GB of VRAM. That's not exactly trivial.I could see it being feasible to get a Qwen-3.6-27b type of model done on something like this. Qwen-3.6-27b at 18tok/s would be a game changer.
trollbridge 34 minutes ago The chatjimmy demo is using a model that needs 6-18GB of VRAM. That's not exactly trivial.I could see it being feasible to get a Qwen-3.6-27b type of model done on something like this. Qwen-3.6-27b at 18tok/s would be a game changer.
The chatjimmy demo is using a model that needs 6-18GB of VRAM. That's not exactly trivial.
I could see it being feasible to get a Qwen-3.6-27b type of model done on something like this. Qwen-3.6-27b at 18tok/s would be a game changer.