← Back to context Comment by adastra22 2 months ago I’m confused as to why you think a GPU is necessary? It’s just linear algebra. 3 comments adastra22 Reply oreoftw 2 months ago most likely he was referring the fact that you need plenty of GPU-fast memory to keep the model, and GPU cards have it. adastra22 2 months ago There is nothing magical about GPU memory though. It’s just faster. But people have been doing CPU inference since the first llama code came out.
oreoftw 2 months ago most likely he was referring the fact that you need plenty of GPU-fast memory to keep the model, and GPU cards have it. adastra22 2 months ago There is nothing magical about GPU memory though. It’s just faster. But people have been doing CPU inference since the first llama code came out.
adastra22 2 months ago There is nothing magical about GPU memory though. It’s just faster. But people have been doing CPU inference since the first llama code came out.
most likely he was referring the fact that you need plenty of GPU-fast memory to keep the model, and GPU cards have it.
There is nothing magical about GPU memory though. It’s just faster. But people have been doing CPU inference since the first llama code came out.