This does concern me a bit but I am pretty careful and have it set up on multiple devices where possible so that if one gets damaged or lost the other will still work.
I use a password manager and two factor authentication wherever available. I don’t trust websites enough to be using the same password across multiple of them.