Comment by dns_snek

Yes something like that has been done, the solution uses asymmetric cryptography instead of symmetric and it protects the entire communication stream instead of just opening the gates - it's called Wireguard.

I jest, but you're describing authentication protocol which uses the UDP destination port field to carry each character of the "one time password" in a roundabout way. You could simply send 1 packet containing the password to the daemon.

I've never seen it used in the wild but apparently, it exists: https://github.com/mrash/fwknop