Comment by TeMPOraL

> should add the other phone as a second auth method on every site.

That's the problem right there. Migrating my phone recently (without having broken/bricked the previous one, which is somehow even worse wrt. transferring 2FA these days than getting new phone after old one breaks!), I discovered that most sites I used did not allow more than one authenticator app. If I try to add new phone as second-factor auth method, the website deletes the entry for the old phone.