Artificial intelligence (AI) has become far more believable, with a recent example being the viral image of the Pope in a puffy jacket fooling many.
But images aren’t the only pieces of content that AI can produce convincingly—AI voice cloning, which essentially mimics one’s voice for different use cases, is also emerging. An example is the imitation of President Joe Biden’s voice during his State of the Union address.
It’s proving to be another significant tool with high business potential but can raise ethical and legal concerns.
This article will give you a glimpse of how voice cloning AI works, the concerns you need to be aware of, and what you can expect from the future of this technology. As a bonus, we’ll show you a few apps you can use to clone your voice—ethically, of course.
What is AI Voice Cloning?
Voice cloning AI uses artificial intelligence software to create a near-identical voice of a speaker, mimicking everything from syllable pronunciations to intonation patterns.
It differs from speech synthesis, where AI uses different, pre-defined voices to replace speech. Both are often used together since once you have an AI voice clone, you can use it to say whatever you want in any language or emotion.
The audio clip the voice cloning software needs to learn from doesn’t have to be long. McAfee reports that a three-second voice clip is long enough for the AI to learn and copy the speaker.
The potential for this technology is staggering, as governments worldwide are still determining the best laws and regulations to place for its safe usage. In the meantime, many people have already begun experimenting with voice clone AI tools for various purposes.
The Current State of AI Voice Cloning
Just as with chat-based and photo-generating AI, people worldwide are still figuring out how to use clone voice AI tools best. Below is a general overview of how people and businesses use AI voice cloning.
Rise of free AI voice cloning software
The ability to clone voice isn’t exclusive to the tech-savvy or super-rich. Since numerous businesses now offer voice cloning software at varying prices, it isn’t surprising that the market is set to grow at a compound annual growth rate (CAGR) of 17.2% by 2028.
Of course, the output quality may not be as convincing as paid services, but the number of businesses offering the type of software underscores the demand.
Replicating celebrity voices
Mimicking celebrity voices has become voice cloning AI’s most popular use case, pushing creative boundaries and drawing potential legal issues. Many famous people, including Taylor Swift, Joe Rogan, and former US presidents, have been victims of AI voice cloning.
A significant recent example occurred in April 2023, when TikTok user ghostwriter977 released the song “Heart on my Sleeve,” using the voices of international artists Drake and The Weeknd—despite none actually singing or being connected to the project in any capacity.
Many argue that it’s the first viral AI-generated song, receiving over 230,000 views on YouTube and 625,000 streams on Spotify before copyright strikes from Universal Music Group—the artists’ label—took it down.
Modern-day celebrities aren’t the only ones getting their voices replicated.
The filmmakers of the documentary The Andy Warhol Diaries used software to create a synthetic voice of the famed pop artist, Andy Warhol, to narrate portions of his diary, bringing his voice back to life and highlighting how technology has the potential to preserve someone’s identity long after they’ve passed.
Provides more accessibility for people with disabilities
One of the more practical use cases of voice cloning AI is to help those at risk of losing their voice or ability to speak due to health complications, such as those recently diagnosed with ALS (amyotrophic lateral sclerosis), to preserve their voice.
One example is Apple’s Personal Voice feature, which the brand previewed in May 2023. The software enables users to create a synthetic voice that their family and friends can recognize. All they need to do is read parts of randomized texts aloud for 15 minutes for the software to learn and replicate their vocal profile accurately.
A similar service, CoeFont, is being developed in Japan that even offers free use for people who have difficulty speaking, such as those who stutter or are diagnosed with Dysphonia. They’ve reported that more than 400 users have used their service since launching in May 2023.
Dubbing and localizing content
More businesses understand the need for localized content in the globalized world, especially since roughly seven in ten consumers (68%) say they would switch to a brand that offers content in their native language.
The traditional method to localize content would be to hire a translator or foreign voice actors to dub the content. However, thanks to innovations in technology, that may be optional.
AI dubbing is becoming an emerging trend to allow content creators and production companies to dub their content for various international markets without hiring foreign voiceover artists. Entertainment companies can now release series, movies, and songs in different languages to appeal to the local audience.
An example is K-pop artist Midnatt releasing his song, “Masquerade,” in English and using voice AI to release versions in six languages. Viewers watching the music video on YouTube can click Settings to change the audio track to their language to hear the difference.
His record label was even able to synthesize his voice as a woman so he could feature on his own song, presenting vast creative opportunities for solo musicians.
Contributing to scams
Creating a synthetic voice has allowed cybercriminals to scam unassuming victims, as Jennifer DeStefano, a mother in Arizona, experienced early in April 2023. She received a worrying phone call from who she believed to be her daughter, crying and saying that she’s been kidnapped and the criminals demanded a ransom. However, her daughter was safe all along.
The US Federal Trade Commission (FTC) said that AI has allowed scammers to enhance their family emergency schemes, making it sound far more convincing to you to hear a loved one say they’re in trouble. In these moments, some experts recommend agreeing on an “AI safeword” with your loved ones to ensure the voice is actually from them.
Regardless, the negative implications of publicly-accessible AI voice cloning software are clear. Many people have raised ethical and legal concerns about this technology that you should be aware of if you intend to clone your voice.
Ethical and Legal Concerns of Voice Cloning AI
Governing bodies, businesses, and users are still working to understand all the ethical and legal concerns that an AI voice clone can bring. Although still a nascent technology, below are some prevalent issues that should be aware of.
Consent and privacy implications
The ease with which scammers can train voice cloning software to learn specific voices puts content creators and musicians at risk of fraud and impersonation. These instances put into question whether or not artists and content creators should copyright their voices.
Additionally, people’s livelihoods are threatened since the potential for identity theft is much higher.
It threatens privacy and cybersecurity by allowing criminals to bypass voice-based authentication systems. This occurred to Centrelink and the Australian Taxation Office (ATO), where criminals used a synthetic voice to fool the voiceprint security systems meant to verify identities through voice recognition.
Misinformation and manipulation
AI deepfakes continue to be a hot topic of discussion, threatening to divide and manipulate communities. The concern is that voice cloning AI is evolving to become highly convincing faster than governments can regulate it.
It can manipulate and affect a celebrity’s reputation if online trolls release audio of a celebrity spewing offensive comments or jokes; a recent example is the voice of British actress Emma Watson reading Hitler’s Mein Kampf.
Impact on human voice actors and job displacement
Many people worry about their job security as AI becomes more and more capable of performing traditionally human tasks. Voice cloning AI threatens voice actors, especially.
There have already been incidents where voice actors have been shocked to find AI copying their vocals for people to use for their projects. In February 2023, a few video game voice actors publicly condemned contracts they received, requiring them to sign away their voices to AI.
What will become of voice actors if it’s become much easier to use AI to narrate or dub content for you? It’s a question worth looking into since it can potentially cause thousands of voice actors their job.
The state of AI voice cloning is complicated. Many are still experimenting with the technology. Given that, it’s worthwhile to consider what the future holds for voice cloning AI.
What You Can Expect with AI Voice Cloning
Nothing about voice cloning AI is set. As the world continues to understand and discover the possibilities of this technology, it’s best to consider what the future might hold for it.
1. Tighter government regulation and broader ethical discussions
Governments will likely impose stricter regulations on using voice cloning AI. Senator Richard Blumenthal highlighted how convincing voice cloning software has become by making the technology recite his opening statement at a recent US Senate hearing.
What might these regulations and policies include? They may address whose voices can be cloned through AI and define the exact purposes of the technology. It could stipulate that companies must disclose whether or not they use voice AI for any of their processes. Additionally, courts must still determine who owns the rights to an AI-generated voice.
These legal parameters could help people protect against the risks and dangers of clone voice AI.
2. Increased use for content creation
There are ethical uses for voice cloning apps. For instance, voice cloning software can be a productivity tool if you’re a video content creator producing faceless YouTube content. Training the AI to mimic your voice can significantly reduce production time since you won’t have to spend hours recording and re-recording audio in front of a microphone anymore.
Another is AI marketing, which allows you to leverage AI to produce materials at a much faster rate and lower cost than before.
3. More AI detectors
With how convincing AI has become, the ability to discern whether a piece of content is authentically human is crucial to avoid falling for misinformation. You can expect to see more people create more reliable detectors to ensure that, no matter how convincing a piece of content is, everything you consume is made by a human.
4. Greater popularity of AI voice in the entertainment industry
The filmmaking industry is becoming increasingly comfortable with AI dubbing, with the Motion Picture Association (MPA) recently awarding certification to the AI dubbing startup Deepdub. This title ensures that the startup’s AI can meet the high standards of the entertainment industry.
Deepdub isn’t alone in offering AI services to the entertainment industry. Many venture capitalists have begun investing in numerous AI startups to bring AI to the movie production companies like Netflix, Marvel, and Lucasfilm.
In a similar development, AI company Flawless announced in May 2023 that they’re partnering with US and UK distributors to release English versions of non-English movies to different regions, dubbed and lip-synced by AI.
With experts expecting the industry to be worth $416.8 billion by 2030, AI is poised to become more integrated to produce more high-quality content for streaming services.
Popular Voice Cloning Apps
If you want to clone your voice through software, here are a few popular tools you can check out.
Resemble.AI
Resemble.AI offers various products and services to help you create a synthetic voice you’re satisfied with. For instance, if you want to replace a few words in your recorded audio without re-recording, their Resemble Fill feature will help edit the clip seamlessly.
They also have a Custom AI Voices API developers can integrate into various tools they already use. Their voice cloning AI will only need at least three minutes of audio or speaking 25 predetermined sentences to learn voices.
BeyondWords
BeyondWords has a library of over 550 AI voices in more than 140 languages that are ethically created; the company collaborates with voice actors through its Voice Cloning Contract. They also use Natural Language Processing (NLP) to analyze user text and convert it into authentic-sounding speech.
Respeecher
Respeecher prides itself on allowing content creators, filmmakers, and game developers to create synthetic voices. Notably, they’ve worked with companies such as Lucasfilms to generate an AI-generated voice for an older actor reprising his younger role and Mondelez International to produce highly targeted and localized marketing.
The company uses both digital signal processing algorithms and a deep generative model to allow its artificial intelligence to learn and mimic not only the voice but also the emotions and delivery of passages.
Eleven Labs
Many know Eleven Labs for their library of celebrity voices, which you can readily use for your content with their VoiceLab product. They showcased their expertise by dubbing Leonardo DiCaprio’s speech at the United Nations with other celebrities, such as Joe Rogan and Steve Jobs.
The company aims to generate realistic-sounding voices with its AI model focused on capturing logic and emotions in texts with its Speech Synthesis platform. It gathers context about each sentence and paragraph to understand how to intonate and speak convincingly.
PlayHT
PlatHT has a library of voices you can clone for your projects, from Elon Musk and Neil DeGrasse Tyson to John F. Kennedy and Barack Obama. Their real-time Voice Cloning software allows you to create a synthetic voice that captures the speaking style and preserves the subject’s accent and speaking nuances.
Their voice cloning AI will require at least an hour of clear speaking audio to kickstart its vocal analysis and learning process.
It’s important to note that all these companies have outlined the ethics behind their products, which you may view on their website.
Keeping Your Ear to the Ground
AI voice cloning can have considerable effects on society, both positive and negative. While on the one hand, businesses can use technology to help people continue “speaking” long after losing their voices to medical conditions or creators to reduce their production time.
That said, voice cloning AI isn’t completely free from ethical or legal concerns. Fraudsters may still use it to impersonate people in family emergency schemes or to bypass voice authenticators to access highly sensitive and confidential data.
While governments continue to discuss possible laws and policies around appropriate AI use, it’s up to private companies to use it more responsibly. That includes following cybersecurity best practices, such as asking for consent for access to consumer data and practicing transparency with how you use the technology.
Voice cloning AI continues to evolve. Staying updated on its latest developments can help you understand how best to use the technology to deliver value to your customers that AI can’t replicate.
Ready to learn more? Let’s talk.
