Comment by diath

Not sure why you're getting downvoted as from security standpoint that would be the right thing to do; but actual good server providers take it up a notch and allow you to upload the public key through their admin panel during server set up and the password you get only works locally for sudo etc. There's no reason why a fresh Linux setup should ever offer a publicly available SSH server with password based auth out of the box.