Comment by gorkish
5 hours ago
This is a ConnextX4 on a thunderbolt bridge. This is well trodden territory. Unless some hero of open source has come to bat recently, MacOS doesn't support RDMA with this card or over Ethernet at all for that matter. It's only supported over Thunderbolt.
His iperf3 results show that it's able to go full beans on a single 25g port. A cross platform SMB file copy is kind of a meaningless benchmark in this context.
IMO a pretty cool product would be a network switch providing native thunderbolt interfaces to multiple upstream hosts. That wouldn't magically fix this RDMA issue, though. This is just a separate thought. Seems like the kind of thing Mikrotik should make.
Commented on another top comment but yeah, I've been running a similar setup for about $200 all in. Using the PCIe LX version takes care of much of the heat issue, and an eGPU dock (Razer Core in my case) takes care of the fan. Craigslist and eBay were the sources.
Speeds aren't any better, but at least it was cheap and fiber.