Comment by toast0
8 days ago
> Yeah I could get higher-quality 10G cards and put them in all the machines but they seem expensive...
Bulk buying is probably hard, but ex-enterprise Intel 10G on eBay tends to be pretty inexpensive. Dual spf+ x520 cards are regularly available for $10. Dual 10g-base-t x540 cards run a bit more, with more variance, $15-$25. No 2.5/5Gb support, but my 10g network equipment can't do those speeds either, so no big deal. These are almost all x8 cards, so you need a slot that can accomidate them, but x4 electrical should be fine (I've seen reports that some enterprise gear has trouble working properly in x1/x4 slots beyond bandwidth restrictions which shouldn't be a problem; if a dual port card needs x8 and you only have x4 and only use a single port, that should be fine)
I think all of mine can pxeboot, but sometimes you have to fiddle with the eeprom tools, and they might be legacy only, no uefi pxe, but that's fine for me.
And you usually have to be ok with running them with no brackets, cause they usually come with low profile brackets only.
+1 ebay x520 cards. My entire 10g sfp+ home network runs on a bunch of x520s, fs.com DACs/AOCs, Mikrotik switches, and an old desktop running FreeBSD with a few x520s in it as the core router. Very very cheap to assemble and has been absolutely bulletproof. IME at this point in time the ixgbe driver is extremely stable.
x520s with full-height brackets do exist (I have a box full of them), but you may pay like $3-5/ea more than the more common lo-pro bracket ones. If you're willing to pop the bracket off, you can also find full-height brackets standalone and install your own.
Also, in general: in my experience avoiding 10gbe rj45 is very worthwhile. More expensive, more power consumption, more heat generation. If you can stick a sfp+ card in something, do it. IMO 10gbe rj45 is only worthwhile when you've got a device that supports it but can't easily take a pcie nic, like some intel NUCs.
sfp+ is clearly cheaper, and less heat/power, but I've got cat5e in the walls and between my house and detached garage, so I've got to use 10g-baseT to get between the garage and the house, and up to my office from the basement. At my two network closet areas, I use sfp+ for servers.
I think my muni fiber install happening this week might have a 10G-baseT handoff, and I've got a port for that open on my switch in the garage. If that works out, that will be neat, but I'll need to upgrade some more stuff to make full use of that.
Oh true, good point, being wired for ethernet is another valid usecase. I'm lucky in that my ONT is just a commodity Nokia switch I can slap any sfp+ form factor transceiver I want in the appropriate port of for the connection to the router, so in my case 10gbe is truly banishable to the devices I can't get a pcie card into. I'm still in the phase of masking taping cables to the ceiling instead of doing real wall pulls, but when I do get around to that I feel like I'm going to pick up an aliexpress fiber splicer and pull single-mode fiber to futureproof it and make sure I never have to deal with pulls again (and not be stuck on an old ethernet standard in the magical future where I can get a 100gbit wan link).