Comment by willglynn
3 years ago
In 2024 I would suggest deploying 2x25G instead, via e.g. MCX4121. Pricing is similar (<$30 NICs), but:
* 2x25G throughput is higher than 40G,
* 25G latency is lower than 40G,
* you can use 25G ports as 10G ports, and
* you can use DACs to connect 4x25G <=> 100G
That last point is particularly relevant given the existence of switches like the Mikrotik CRS504, providing 4x100G ports on 25W.
those are all reasonable points, if I were doing mine again I would spend a little more and go up to 100gbe
if you run all older mellanox gear the cx3 can do the kinda nonstandard 56gbe as well
What are you doing that you need 100gbe?
I am still on 1gbe... I guess I don't transfer anything bigger than a few GiB time to time.
No individual connection other than like a cr tral storage server needs 100gbe, at least for me, but a 100gbe backplane is good for a lot of 1gbe poe devices as an example. With residential fiber/coax reaching 5gb, 1gb is not enough.