Yeah, it is indeed scary that machines can mimic human voice perfectly. And super creepy when a machine can talk exactly like or mimic an individual. But it feels terrifying that this machine or AI will be owned by a big corporation or government and not you.
Already a company like Google has a huge data set on any person's voice who has been connected to the internet on smart devices with their apps on by default. And it would be easy for them to put together a personalized AI that talks like you.
Governments must act soon and enact laws that will prevent the use of such voice data by corporations if an individual wishes so. Any person must be able to send a notice to Google to take down all their personal data including voice and Google must comply by purging such data across all geographies!