← Back to context

Comment by Palomides

3 years ago

not really true, a similar setup with 40Gbe (three nodes, no switch) would run you less than $100 for pcie cards and cables

21 comments

Palomides

Reply
hamandcheese  3 years ago

The only NICs I see in that price range are old Mellanox cards.

Intel NICs are 5-10x that price. I'm not sure why, but my suspicion is that it has to do with driver support and/or interoperability.

nickstinemates  3 years ago

Can you provide links? I'll upgrade my home lab from 10g to 40g immediately at that price.

  • willglynn  3 years ago

    In 2024 I would suggest deploying 2x25G instead, via e.g. MCX4121. Pricing is similar (<$30 NICs), but:

    * 2x25G throughput is higher than 40G,

    * 25G latency is lower than 40G,

    * you can use 25G ports as 10G ports, and

    * you can use DACs to connect 4x25G <=> 100G

    That last point is particularly relevant given the existence of switches like the Mikrotik CRS504, providing 4x100G ports on 25W.

    • Palomides  3 years ago

      those are all reasonable points, if I were doing mine again I would spend a little more and go up to 100gbe

      if you run all older mellanox gear the cx3 can do the kinda nonstandard 56gbe as well

      2 replies →