Home Education They Dedicated Their Lives to Teaching. Then the...
Education

They Dedicated Their Lives to Teaching. Then the Deepfakes Started

They Dedicated Their Lives to Teaching. Then the Deepfakes Started
Key Points

For Luis DeSantiago, a substitute teacher in Los Banos, California, it started with a few text messages. On March 12, as he was working his other part-time job at a coffee shop, his phone lit up. A fellow barista said there was an AI photo circulating on social media.

For Luis DeSantiago, a substitute teacher in Los Banos, California, it started with a few text messages. On March 12, as he was working his other part-time job at a coffee shop, his phone lit up. A fellow barista said there was an AI photo circulating on social media. Her cousin wanted to know if it was DeSantiago. At first, DeSantiago wasn’t worried—he figured that inappropriate photos violated TikTok’s community guidelines and were usually quickly removed. “When I finally saw the image, that’s when it felt like a punch to the gut,” DeSantiago tells WIRED. Parts of it resembled a mirror selfie that he remembered taking at a job in college. But instead of his work uniform, he was wearing lingerie. And instead of standing normally, as he had been in the original photo, he was posing suggestively. The texts kicked off a monthslong saga that crossed two schools and involved students and administrators. And DeSantiago is far from the only educator who’s been victimized this way. WIRED spoke with three other teachers who allege that their students made sexualized photos and videos of them, likely using AI. They say the incidents have been profoundly distressing and disappointing—and in some cases, it’s led them to reassess how they view their work and their relationships with their students. “This Could Include Other Students” After he received the texts, DeSantiago began to piece together who might have been behind the image. One account that shared it used their real name in their TikTok profile—it was a student DeSantiago knew from the local middle school, where he regularly filled in. DeSantiago suspected the involvement of two other students based on people who followed and interacted with that account, and his knowledge of student friend groups. He was scheduled to work at that very middle school the next day. DeSantiago reported the incident to the assistant principal, who assured him that the school would "take care of it.” Later that evening, he decided to look for the image by searching his name on TikTok. He discovered that a different student had created an account impersonating him—it used his name, slightly mispelled, and had the same photo as their profile picture. Over the next few weeks, DeSantiago says, he went back and forth with school administrators, the parents of the students, and even the police. “If these students aren't afraid of generating and posting these images of me, I am concerned of what they may be circulating privately through text messages and group chats,” DeSantiago wrote to the principal. “Keep in mind this could include other students as well.” One student, who admitted to creating the image, was disciplined with in-school suspension. DeSantiago was later told that the image was also shared on Snapchat. But DeSantiago says the image continued to proliferate, and other students began posting it. He says he also received several crude, sexual comments on his TikTok account. DeSantiago told administrators he didn’t want to take assignments at the middle school “until this matter is fully addressed.” The district did not respond to WIRED’s request for comment. He took a substitute assignment at the local high school, but the problems followed him. During one class, he received an AirDropped photo of the same AI-generated image before it was quickly revoked. He could tell it was circulating among the class. The principal told him the school was launching an “immediate investigation” and offered to meet with DeSantiago or make the school counselor available, but DeSantiago says he probably won’t return to the district this year. He tells WIRED that, at least right now, he doesn’t feel safe doing so. “It Makes You Question Everything” When fake nude images of nonconsenting adults are disseminated, it could constitute cyberstalking and violate the federal Take It Down Act, which makes it a crime to publish nonconsensual “intimate visual depictions of individuals, both authentic and computer-generated,” on the internet. In most of the accounts shared with WIRED, the teachers didn't directly report the content to the platforms on which it appeared. Snap spokesperson Monique Bellamy shared the company's policies on harassment and sexual content, which prohibit sharing AI-generated or manipulated content to humiliate or sexually harass others. She told WIRED that the company has "established processes" for reporting such content, and that it continues to "develop features and tools designed to support the safety, privacy, and well-being of our community." TikTok did not provide comment before publication. Donna Ballman, a retired employment advocacy lawyer, tells WIRED that if a student creates and shares sexual images of a teacher, it constitutes a “clear case” of sexual harassment in the workplace. Ballman says that under federal antidiscrimination laws, employers have an obligation to create a workplace free of sexual harassment. In a school, this could mean disciplining the student or moving them to a different classroom. She adds, however, that there's “no blueprint” for how schools need to respond to these incidents. Administrators don’t even have to tell a victimized teacher how a student may have been disciplined. “They just have to make [and] create a safe environment for the victim,” Ballman says. One middle school teacher in her mid-twenties tells WIRED that she’s seen so much AI-generated content used to harass teachers that she’s bracing for the next wave. (She requested anonymity because she did not have permission from her school district to speak about her experience.) Last year, she says, students found public photos of a teacher in her sixties, manipulated them to make it appear that she was committing sex acts, and shared them with each other in a Snapchat group chat. A few months ago, the teacher says, she learned from a student about a TikTok account impersonating her. It looked AI-generated—her face had the characteristic smoothness of AI-generated content, her mouth was moving unnaturally while speaking, and the sound of her voice didn’t match her own. Some of the videos showed her doing things like skydiving or backflips. But others depicted her talking about sexual acts she performed and enjoyed. She says she told her principal about the account and that they assured her that they would “see what we can do.” Throughout the day, students continued to warn her about the account. She says that some told her that they knew who made the account—one of their classmates. She says the student was known to have behavioral problems. “I know a lot of the kids that reported it to me genuinely believed that was my account, and that I was saying those things,” she tells WIRED. “I was like, ‘Oh my god, you guys—that is not me.’” She asked them to report it to TikTok, and she says the account was quickly taken down. Still unnerved by the incident, she mentioned the TikTok account during the next monthly teacher meeting. A colleague said he had seen a seemingly AI-generated fake video of himself getting arrested, which he believed was generated by the same student. As days and weeks passed, she says, she became the target of escalating, in-person harassment from this same student, who began “cursing me out, calling me names, telling me to fuck off, calling me a bitch.” She raised it at the next faculty meeting, and the assistant principal said that the same student was calling and texting them in the middle of the night and posting apparent AI-generated photos of them to TikTok. She says the principal removed the student from her class and tried to limit his access to her. But she says that the harassment continued until the student ultimately left the district, and TikTok removed the account. She says that these behavioral issues aren’t exclusive to this particular student. “It's kind of unsurprising that we are dealing with that, because these kids just have a hard time,” she says. "I'm just worried about this happening again.” Other teachers say their experiences have fundamentally changed the way they relate to students. Alice, who asked to be identified by her first name, works for a UK college that trains students aged 16 to 19 in trades like construction. Her job combines administrative responsibilities, like taking attendance, with the role of a college counselor. She says she individually meets with about 150 students throughout the year and takes pride in being able to be a sounding board, give them advice, and serve as a mentor. This winter, she says, school administrators told her that a student filed a report claiming two videos were circulating among his classmates. One appeared to be real, and it showed her having a conversation with a student. The other one, they said, appeared to be a manipulated version of the first. But instead of talking to the student, it showed her taking her clothes off and dancing around in what was either underwear or a bikini. Alice tells WIRED that as a qualified youth worker, part of her job often involves talking about things that students may feel “too embarrassed to tell their parents about,” like drugs, parties, or having safe sex. She says she had several of those conversations with the student she suspected had created the video. The student denied it when questioned by administrators and soon dropped out of the school. (The college did not respond to WIRED’s request for comment.) “It makes me reflect as well on those conversations, thinking, did he think that he could overstep a line because we were having these discussions?” she says. “It makes you question everything,” she says. Afterward, the college gave her permission to step away from the class for a few weeks and took steps to educate students about the danger of creating or distributing deepfakes. But when she returned, she says, the atmosphere had chilled. “I did have a really good rapport with my students—I do have a good rapport with my students,” she says. And she still loves her work. “But it's definitely made me a bit more cautious, a bit more vigilant when it comes to folks.” “It Crosses the Line” Like DeSantiago, Arizona-based sixth-grade teacher Luke Red says his students circulated an AI-generated photo of him in the comments of his public TikTok account. The image depicted him and another male teacher kissing each other. Red says he reported the accounts to TikTok—not for content violations but for being under the minimum age of 13—but says TikTok didn’t remove the accounts. He says that the school immediately launched an investigation and quickly identified the student, who admitted to creating and sharing the image. The student received a three-day suspension—the most severe punishment kids their age can receive for a first-time sexual harassment offense, according to the school district handbook. The district did not respond to WIRED’s request for comment. Red tells WIRED that reflecting on the experience “has kind of sent me into a spiral.” His concerns, he says, are not necessarily about what happened to him. They’re more focused on the enthusiastic, permissive culture surrounding AI use among his students. “It just got to the point where students felt comfortable enough creating inappropriate images,” Red says. “Obviously when you're making your teachers do sexual things with these images, that's where it crosses the line.” Ultimately, he feels more sorry for the students than for himself. “It's just clear [that] so many kids nowadays just don't understand not only how what they're doing can affect others, but just what happens on the internet and how dangerous it can be,” Red says.
Luis DeSantiago (PERSON) Los Banos (LOCATION) California (LOCATION) AI (ORG) DeSantiago (PERSON) TikTok (ORG) Snapchat (PERSON)
Originally published by Wired Read original →