Comment by NamlchakKhandro
4 hours ago
48gb of vram.
a machine like this is about a years rent for most people.
a small car for most others.
4 hours ago
48gb of vram.
a machine like this is about a years rent for most people.
a small car for most others.
I think he’s talking about the Mac Mini unified memory.
48G RAM is pretty useful if you want to run k8s locally for tests / exploration
true but if you're actually running k8s and similar workloads, chances are it might eat memory that LLM requires.
you'll also notice these articles rarely specify their context window in tokens, because it is small, usually 30k to 70k tokens and it gets slower as it fills up.