Comment by nickstinemates
3 years ago
Incredible. Traditional 10Gb network gear is very expensive to buy and to operate. $50 is a lot cheaper.
3 years ago
Incredible. Traditional 10Gb network gear is very expensive to buy and to operate. $50 is a lot cheaper.
not really true, a similar setup with 40Gbe (three nodes, no switch) would run you less than $100 for pcie cards and cables
The only NICs I see in that price range are old Mellanox cards.
Intel NICs are 5-10x that price. I'm not sure why, but my suspicion is that it has to do with driver support and/or interoperability.
I run mellanox connectx3 cards, they work immediately with no extra drivers on windows 10/11 and every linux I've tried
mellanox is/was quite good at getting code upstreamed
maybe I need to do my own blog post about my pile of computers...
3 replies →
Mellanox works fine.
6 replies →
Can you provide links? I'll upgrade my home lab from 10g to 40g immediately at that price.
In 2024 I would suggest deploying 2x25G instead, via e.g. MCX4121. Pricing is similar (<$30 NICs), but:
* 2x25G throughput is higher than 40G,
* 25G latency is lower than 40G,
* you can use 25G ports as 10G ports, and
* you can use DACs to connect 4x25G <=> 100G
That last point is particularly relevant given the existence of switches like the Mikrotik CRS504, providing 4x100G ports on 25W.
3 replies →
Mellanox connect-x 3 probably.
cx353a and cx354a, prefer the fcbt models, but you can reflash them all to fcbt
2 replies →