Comment by angoragoats

> I think it's bad form to say "citation needed" when your original claim didn't include citations.

I apologize, but using multiple GPUs for inference (without any sort of “daisy chaining”) is something that’s been supported in most LLM tooling for a long time.

> Regardless - there's a difference between training and inference.

No one brought up training vs. inference to my knowledge, besides you — I was assuming the machine was for inference, because my experience building a machine like the one I described was in order to do inference. If you want to train models, I know less about that, but I’m pretty sure the tooling does easily support multiple GPUs.

> And pytorch doesn't magically make 5 gpus behave like 1 gpu.

I never said it was magic, I just said it was supported, which it is.