Comment by khanhnguyen8386
8 hours ago
Can't wait to run this at 0.02 tokens/sec on my CPU so I can get a response just in time for next month.
8 hours ago
Can't wait to run this at 0.02 tokens/sec on my CPU so I can get a response just in time for next month.
Unfortunately I don't think modern consumer CPUs are physically capable of addressing enough RAM to even load the model into memory. We'd have to wait for some random person to make an extremely quantized version before we could reach those blazing speeds