Comment by Karliss

USB HID protocol has been designed with huge flexibility and self descriptive devices in mind so that in theory you shouldn't require a custom drivers for vast majority of current input devices even controllers order of magnitude more complex than anything currently sells. Just like you don't need a custom driver for each usb flash driver.

In practice half-assed HID drivers by OS, badly designed OS<->application APIs, hardware manufacturers copy pasting HID descriptors from other devices, not following the standard properly, firmware bugs getting fixed with drivers instead of firmware fixes, intentional discrepancies from standard, console manufacturers reinventing the wheel has lead to the current mess.