Hide Anonymize button for certain users

I want to hide the Anonymize button on the Profile page but still give the users the possibility to change their data (eg password change). I know the ‘Update Self’ permission, but this is not what I want.

In my opinion, the button is a bit risky and should only be reserved for administrators.  Did I miss something or isn’t it possible?

Thanks!