Comment by dgacmu

Somewhat related, I recently wanted to fix a slow nfs problem (1 server, 1 client -- basically, a file server and a machine with some GPUs). It turns out that used previous generation mellanox 100gig NICs are quite cheap now, and you can direct-connect two machines for about $600.

(What I really wanted was the low latency, but the bandwidth is handy to have sometimes)