Comment by huggingmouth

Matrix clients typically complain very loudly when keys change or unverified sessions get added to your chats.

Id's say it would mitigate it A LOT. Though I have to agree that there is still a lot of room for improvement.

I personally prefer the approach taken by briar: your public key is your address. Pretty cut and dry, not much room for shenanigans.