TikTok mistakenly released a version of its AI video generator that let avatars recite Adolf Hitler’s Mein Kampf and Osama bin Laden’s “Letter to America” on command.
Last week, TikTok announced its new tool “Symphony Digital Avatars” which enables brands to use AI-generated versions of real people in ads to give a “human touch” to campaigns.
NEW: TikTok released an AI video generator. We found it recited “Mein Kampf” and Osama Bin Laden on command with no watermarks. It pulled access after CNN asked them about it. https://t.co/6RRtPl6s0v pic.twitter.com/hubruBVtDx
— Jon Sarlin (@jonsarlin) June 21, 2024
Symphony Digital Avatars allowed advertisers to use AI-powered dubbing so that brands could enter a script and make the avatars say what they want within TikTok’s guidelines in product campaigns. Only users with a TikTok Ads Manager account were supposed to be able to access the tool.
TikTok placed zero content restrictions on the app. Any words we put in created a video with a peppy “Stock AI” Avatar reciting them. That included:
-The 14 words
-An excerpt from Mein Kampf
-Osama Bin Laden’s Letter to America
-A video telling people to drink bleach from “a… pic.twitter.com/I2QbqbDFnw— Jon Sarlin (@jonsarlin) June 21, 2024
However, CNN discovered that TikTok had accidentally posted a link to an internal version of its new AI avatar tool which had zero guardrails or safety measures in place.
Any TikTok user could create unmoderated and unmarked videos which let AI avatars say just about anything — including the ability to recite an excerpt from Adolf Hitler’s Mein Kampf on command.
TikTok did not respond to a follow up about what they are doing to prevent a mistake of this magnitude from happening again
While TikTok is pushing their AI to businesses, it posting an internal testing model into the wild raises serious questions about their internal AI… pic.twitter.com/av9UtbdP3Z
— Jon Sarlin (@jonsarlin) June 21, 2024
CNN tech reporter Jon Sarlin says he was able to access the AI tool with his personal TikTok account. On X (the platform formerly known as Twitter), Sarlin revealed that he was able to get TikTok’s AI avatars to recite Osama bin Laden’s “Letter to America,” say the white supremacist slogan “The Fourteen Words,” tell people to drink bleach and vote on the wrong day, as well as read out Hitler’s Mein Kampf
Furthermore, none of these videos had a watermark to indicate they were AI-generated, meaning such videos could be taken at face value by unsuspecting TikTok users if the clips were to be posted on the platform.
‘A Technical Error’
When CNN reached out to TikTok about these videos, a company spokesperson called it a “technical error” that “allowed an extremely small number of users to create content using an internal testing version of the tool for a few days.” TikTok says it has now resolved the issue.
“If CNN had attempted to upload the harmful content it created, this content would have been rejected for violating our policies,” TikTok adds.
“TikTok is an industry leader in responsible AIGC creation, and we will continue to test and build in the safety mitigations we apply to all TikTok products before public launch.”
On X, Sarlin revealed that TikTok did not respond to follow-up queries from CNN about what the company is doing “to prevent a mistake of this magnitude from happening again.”
Image credits: All photos via X/@JonSarlin.