Comment by CamperBob2

That's a very deep rabbit hole involving PCIe topology on both the hardware and software (NCCL) side, among other things. It's too system-specific to answer directly, but the entrance to said hole can be found at https://github.com/local-inference-lab/rtx6kpro/blob/master/... .

Disregard references to RTX 6000 cards, most of it is generally applicable to all multiple-GPU boxes.