Zoom wants to turn you into an AI-animated, photorealistic avatar — but not before next year.
The announced feature, unveiled today at Zoom’s annual dev conference, will allow a video clip that users record to be turned into a digital clone — complete with head, upper arms and shoulders. Users will be able to enter the text of what they want their digital doppelganger to say, and Zoom will generate audio that will sync with the avatar’s lip movements.
Smita Hashim, Zoom’s chief product manager, told TechCrunch that the custom avatars are designed to help people chat “asynchronously” with colleagues in a “faster and more productive” way.
“Avatars save users valuable time and effort when recording clips, and allow them to increase the scope of video content creation,” Hashim said.
However, they can also pose a risk from deepfake technology.
A number of companies have developed AI technology that digitally “clone” a person’s face and merge that clone with synthetic speech that sounds fairly natural. For example, Tavus helps brands create virtual personas for personalized video ads, while Microsoft launched a service last year that can generate convincing digital replacements for a persona.
However, many of these tools implement specific and strict safeguards to prevent misuse. Tavus requires verbal statements of consent, while Microsoft mandates that their users obtain written permission and consent from any person appearing as an avatar.
Zoom was a little more vague about its security measures.
Citing Zoom’s usage policies that prohibit abuse, Hashim said the company is building “numerous safeguards” into its custom avatars feature, including “advanced authentication” and watermarks.
“We will continue to review and add safeguards as needed in the future,” Hashim said. “We use (…) technology to make it obvious when a clip is avatar-generated, and (…) to ensure the integrity of avatar-generated content.”
Zoom’s digital likenesses fit into CEO Eric Yuan’s larger vision of creating an AI that could one day speak in Zoom meetings, answer emails and take phone calls for you.
But these similarities come at a time when deepfake content is spreading like wildfire on social media, making it difficult to distinguish truth from misinformation.
So far this year, deepfakes featuring President Joe Biden, Taylor Swift, and Vice President Kamala Harris have garnered millions of views and shares. The latest fake generative AI images of destruction and human suffering have flooded the internet in the wake of Hurricane Helena.
Deepfake technology has also been used to target individuals — for example, by impersonating loved ones. Losses related to impersonation fraud topped $1 billion last year, according to the FTC.
How exactly will Zoom prevent scammers from using their tool to generate videos of people saying things they didn’t say for malicious purposes? It’s not clear yet. The mock-up shown by the company shows a visible watermark in the upper right corner of a video clip with a custom avatar. However, such watermarks can easily be cropped using a screen capture tool.
We’re expected to learn more closer to the first half of 2025, when Zoom plans to launch custom avatars for Zoom Clips, its asynchronous video tool, as part of a $12-per-user-per-month premium add-on.
Whatever steps Zoom takes — or doesn’t take — there are ongoing regulatory efforts to try to stem the tide of deepfake content.
In the absence of a law criminalizing deepfakes at the federal level in the US, more than 10 states have passed statutes against counterfeiting with the help of AI technology. The California bill — currently pending — would be the first to allow judges to order deepfakes to take them down or face potential fines.
The post Zoom’s custom AI avatar tool may carry risks appeared first on ITNetwork.
Source: www.itnetwork.rs
