Comment by Daviey

Keeping the key in the same room as the padlock only protects against casual drive theft and secure disposal.

Personally I'm more worried about someone stealing the entire server or a local threat actor.

Sure, keep TPM to help with boot integrity, maybe even a factor for unlock, but things like Clevis+Tang (or Bitlock Network Unlock for our windows brethren) is essential in my opinion.