Comment by mrweasel

The Microsoft Lumia 540 looks remarkably like a modern phone still and it had a fairly easily replaceable battery, because it allowed you to replace the back cover.

There's also the Lumia 920, which is arguably a nicer looking phone than anything Apple current have, also have a fairly easily replaceable battery, requiring you to remove just two screws.