Comment by yubainu
3 days ago
That's amazing. I'm developing sub-tools for LLM as a hobby on an RTX3050 (4GB), but I can only run lightweight models like 1B and 2B. Is it possible to use your tool to make the CPU take over some of the VRAM movement?
3 days ago
That's amazing. I'm developing sub-tools for LLM as a hobby on an RTX3050 (4GB), but I can only run lightweight models like 1B and 2B. Is it possible to use your tool to make the CPU take over some of the VRAM movement?
No comments yet
Contribute on Hacker News ↗