Comment by nucleardog

You can set a password that’s required to authenticate a new device.

Once that’s set, after the SMS code, then (assuming you don’t have access to an existing logged in device because then you are already in…), you can either reset the password via an email confirmation _or_ you can create a new account under that phone number (with no existing history, contacts, etc).

If you set a password and no recovery email, there is no way for them to get access to your contacts or chat history barring getting them from Telegram themselves.