Comment by patrakov

> Maybe the SSH agent on the client can re-authenticate to the server when requested?

There is a PAM module that does this: https://github.com/jbeverly/pam_ssh_agent_auth

Note that this is a bad idea from the security standpoint, as it requires SSH agent forwarding. Which means that, if the remote server is compromised, the attacker can use your SSH agent to log into other servers as you.