Comment by Insimwytim

Versions are different during the upgrade. That's how upgrade is done.

You spin up replica, upgrade it to the target engine version, ensure it's up to date and switch over. [1]

[1] https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_...