Meta Ran Ads That Contained AI-Generated Child Sexual Abuse Imagery

Editor’s note: This article contains descriptions of imagery depicting child sexual abuse. Reader discretion is strongly advised. Over the last nine months, Mark Zuckerberg’s Meta has run dozens of paid ads that include explicit AI-generated child sexual abuse material (CSAM) and images of minors alongside sexually suggestive statements, according to details of the ads shared with WIRED. The ads, which in some cases reached several thousand accounts, were targeted at people living in the United States, United Kingdom, and more than a dozen European countries. More than 50 image and video ads containing abusive content were recently discovered in Meta’s ad library by researchers at the Tech Transparency Project (TTP), with the listings showing that some ads linked out to so-called nudify or undressing apps. Meta’s ad library—a transparency tool that catalogues ads displayed on the company’s platforms—shows the ads running across Facebook, Instagram, Messenger, and/or Threads. The findings are the second time in recent weeks that paid ads linked to child sexual abuse material have been found on Meta’s platforms. “These ads made no effort to mask the images or hide what they were promoting,” Katie Paul, the director of the TTP, an independent watchdog group, tells WIRED. “It’s important to point out that this isn’t content posted by third parties on Facebook or Instagram, these are ads that were reviewed, approved, and allowed to run by Meta, never encountering interference while the company collected the ad dollars.” The ads discovered by the researchers—which have now been removed by Meta for violating its policies on child sexual abuse and exploitation material, and adult sexual solicitation and nudity—were all published between November last year and the start of August. The ads, according to data in Meta’s ad transparency library, often ran over a period of several days and some only targeted men. One video ad, the TTP researchers say, used a thumbnail image of a child sitting on the floor with text appearing on top of it saying: “Realizing Deep Fantasies with Generation AI [sic]. There is so much more than what is shown, use your imagination.” The researchers say that when the ad was clicked, it played video clips of adults involved in sexual acts and then added the child’s face from the thumbnail to a sex act. Another ad the researchers say had been repeatedly posted showed an image of a young girl laying back with her legs spread in a sexual position with text above it saying: “I can show you more.” Most of the ads were not actively being shown to users when researchers found them, but they continued to be available, apparently undetected, in Meta’s ad library for months. Although many of the ads, according to Meta’s data, only “reached” a handful of accounts, at least one reached 2,563 accounts in Europe—including in France, Germany, Ireland, Italy, the Netherlands, Spain, Sweden, and the United Kingdom. Meta’s ad library database does not include details about ad performance in the US or all countries, so the overall reach may be higher. Paul says TTP first discovered the ads last week as its researchers continued to investigate nudify app ads on Meta, after publishing a report about a Chinese advertising partner posting them. Paul says the organization immediately reported the findings to National Center for Missing and Exploited Children’s CyberTipline, which allows people to report online child sexual abuse material. A NCMEC spokesperson says it does not comment on reports it receives. After WIRED reached out to Meta, the company removed the ads from its advertising library. “Sexual exploitation is horrific, and we work aggressively to keep it off our platform,” a Meta spokesperson says. “The majority of these ads had minimal reach and many were disabled before WIRED shared them. Many of these ads also predate new AI technology we launched recently to better detect and block violating ads at upload—and we’re constantly improving these systems. From removing over 36 million pieces of child sexual exploitation content last year to taking legal action against nudify app developers, we will continue to relentlessly fight this abuse.” As the company implies, some of the offending ads got past Meta’s new AI technology. Initially, the TTP researchers found around two dozen ads displaying the abusive content. However, hours ahead of publication, the researchers discovered around 30 more. Many of these ads had been published after WIRED first asked Meta about the content, with multiple ads being live and shown to accounts at the time researchers discovered them. Paul says the tranche included some of the worst instances the researchers had seen. One of the video ads, the researchers say, showed a thumbnail of a girl appearing to put on lip gloss, and when it started playing, the video morphed into the child performing a sex act. Meta’s policies say that all ads are reviewed before they are published, with the system “primarily” using “automated tools” to check them against its policies. The company’s advertising standards stipulate that child sexual and exploitation materials are not permitted, and say that when the company becomes aware of it, it makes reports to NCMEC. Beyond this, Meta’s advertising policies say that ads are not allowed to include any “imagery depicting nudity, sexual activity, depictions of people in explicit or sexually suggestive positions, or activities that are sexually suggestive.” Meta’s ad library now shows that the ads found by the TTP researchers were removed for violating the company’s policies, displaying messages saying they violated rules on “child sexual exploitation, abuse and nudity” and “adult nudity and sexual activity.” In early July, a BBC investigation found that Instagram had been running ads that promoted the sale of what appeared to be child sexual abuse material in India. According to the reporting, the ads used terminology such as “rape video” and linked to channels on Telegram where the illegal content could be purchased. In response, Meta published a lengthy statement saying it takes a “zero tolerance” approach to child sexual exploitation. The TTP’s Paul says that within the cache of ads seen by its researchers, there appeared to be some identical ads that Meta had previously removed due to violations of its policies. “This raises significant questions about how seriously Meta is taking child exploitation in paid advertisements if its own systems can identify identical ads but it does not remove those identical ads for CSAM material,” Paul says, adding that there are no ways within the ad library itself to report content that may violate Meta’s policies when the ads are no longer running. The ads appear to have been mostly posted by Meta accounts with zero or a tiny number of followers; where paying advertisers were disclosed, they were linked to Chinese companies, Paul says. In one instance—the ad showing a girl reclining—the TTP researchers say Meta’s data indicates the advertiser was a Chinese firm called Meet Social. At one point, the company was one of Meta’s ad resellers, which handle ads in China, where Meta’s social media platforms are blocked. According to a 2019 New York Times profile, it published thousands of ads on Facebook per day and expected more than $1 billion in sales. Meet Social did not respond to WIRED’s request for comment. Several of the Meta ads found by the TTP researchers linked to an app called MaskAI that appeared in Apple’s App Store. The app, which was linked to a Chinese software developer, did not show any nudity before it was downloaded, according to a WIRED review of the app. However, once opened, it exclusively contained AI-generated porn and included features that allow users to upload images and face-swap people into sexual situations. After being contacted by WIRED, an Apple spokesperson says that it has removed MaskAI from the App Store for violating its policies against nudification apps, and the company has “always strictly prohibited apps designed to generate, distribute, or consume pornography.” (An email address linked to MaskAI did not respond to a request for comment). Other ads linked to different apps or websites. Since around 2020, as machine learning and subsequent generative AI technology has improved, a large underbelly of dangerous AI nudification and undress services have appeared online—making millions for their creators and causing untold harms to countless women. They have also been used in the creation of AI-generated child sexual abuse material, which is illegal in countries around the world. After WIRED first contacted Meta about the TTP findings, the company asked two online safety organizations to provide comments to WIRED for this story. Sean Litton, the outgoing CEO of the Tech Coalition, a group that works with tech companies to tackle online abuse, said in a statement that Meta has shared “thousands of violating URLs” to the group’s Lantern service, which shares information about abuse content between companies. A spokesperson for the Revenge Porn Helpline in the UK, which works to combat nonconsensual images of adults, also responded with a statement praising Meta’s partnership with StopNCII.org and some of its efforts to combat nonconsensual intimate imagery. “We’ve been clear that nudify apps present a genuinely adversarial problem, not just because of who’s behind them, but because of how quickly this ecosystem adapts around any single point of enforcement,” the Revenge Porn Helpline said. Alexios Mantzarlis, cofounder of digital deception publication Indicator and a former trust and safety worker at Google, says he has found and reported more than 25,000 ads for AI nudifiers on Meta’s platforms in recent years. The company has previously said it has taken down more than 344,000 ads for nudifiers and launched legal action against a Hong Kong company linked to one set of nudifier platforms. (WIRED has previously collaborated with Indicator on reporting about how nudify tech has been used against children in schools around the world). “The actors on the other side are clearly well-equipped to create new advertiser accounts and new domains so that they can keep going, leveraging the same techniques used by scam networks,” Mantzarlis tells WIRED. “Unfortunately, this industry of image-based sexual abuse has gone from a novelty to a permanent fixture of our online ecosystem because of timid enforcement from platforms across the stack.”