GM! We have a few bangers for you today.

First: Can you imagine sitting in jail for days without knowing why, and then finding out that it's because someone hacked CashApp to impersonate you? I would be kind of mad! And elsewhere on 404 this morning, we have some prescient thoughts from an autonomous weapons expert on the state of AI in war, what's going on with cybersecurity experts in the wake of the PHPocalypse, and more insights into how cops view anti-Flock activists.

Let's get into it.

Someone Opened a CashApp In Her Name. She Faces Years in Prison

Amanda Smith answered the door one afternoon in April and came face to face with the local Sheriff’s department in upstate Pennsylvania. The officers said they had a warrant for her arrest for a crime she had allegedly committed more than a thousand miles away in Arkansas.

“I was completely confused,” Smith recalled. For one, she had never been to Arkansas. Second, the warrant was for computer fraud. By her own admission, Smith was not a very tech savvy person.

She had a feeling she was being set up. After discussing with the officers the practicalities of what was about to happen — she was allowed to get her contact lenses — they took her away in handcuffs to the local jail where she spent three nights.

The reason for Smith’s arrest, she learned later, was that a scammer had tricked someone on Facebook Marketplace into sending $1,000 to a CashApp account in her name that the scammer controlled. The scammer created the account using her Social Security number and date of birth. Smith said she has never used CashApp.

What developed over the next few days, and has now stretched across six months, is a Kafkaesque situation in which Smith, in her telling, is being asked to prove her innocence in an ongoing criminal case, while CashApp refuses to provide her any information about the account because she couldn’t prove that she was the owner.

Read the rest of this story here.

MORE FROM 404 MEDIA

“Suspicious behavior:” A federal intelligence center warned police that city council meetings about Flock “may be viewed by extremists as a desirable target,” and told them to be on the lookout for “emotional confrontations” between people who oppose surveillance and government officials, according to a copy of the warning obtained by 404 Media.

“Black October?” For much of the last month, many cybersecurity experts have been warning of an AI-powered “Black Swan” event, saying they’re alarmed by the number of vulnerabilities suddenly discovered, and of the apparent power of AI agents to find and exploit previously unknown bugs. So, what’s really going on? 

The UN met this week to discuss autonomous weapons. “The technology is developing at a pace that is very concerning,” Laurent Gisel, the head of the International Committee of the Red Cross’s (ICRC) legal warfare team, told 404 Media. Gisel was in NYC this week to address the UN about all the things that are driving conflict across the planet. His prepared remarks include a statement on the illegal arms trade, nuclear weapons, mines, and, of course, AI.

FROM AROUND THE WEB 

That’s a W for the Rebel Alliance. In San Diego, where Darth Vader famously spoke in favor of Flock cameras and mass surveillance, the city council just voted to end its contract with Flock. The decision still has to be confirmed, NBC 7 reports, but this is huge news for a city where council members have voted multiple times to keep Flock watching its citizens. 

Near the bank of the Ohio River, a historic American chestnut grove in Mason County, West Virginia has become the epicenter of a fight against a planned 2,250 acre data center campus. Plans for the UK-based startup Nscale’s Monarch Compute Campus, with clients including Anthropic, Nvidia and Microsoft, encompass the former Clements State Nursery and the American chestnut trees still growing there. 

For thousands of years, the American chestnut tree was a staple of life. Its natural range spanned hundreds of millions of acres from Maine to Mississippi, centuries before those names existed. As many as four billion trees provided a hearty nut crop for humans and provided miraculously rot-resistant lumber for railroad ties and housing. It sustained black bears as they prepared to hibernate, fed white-tailed deer and turkey, and shed nutrient-dense leaves. The largest American chestnut trees once grew 100 feet tall and 13 feet wide and represented one-fourth of all hardwoods; a squirrel could scamper along their branches from New England to Georgia without touching the ground, as an old saying goes. 

In 1904, a forester at the Bronx zoo noticed something alarming among their American chestnuts: The trees were dying en masse, struck by a mysterious blight. Within 40 years, that blight — thought to be brought to America on Asian chestnut trees — wiped out the tree’s population to near-extinction. Since then, conservationists have worked to restore American Chestnut trees and breed blight-resistant specimens. Typically, threats to groves include pests, climate change, and the fungus that destroyed their ancestors. Today, however, a new threat has entered the picture: data centers. 

Across the country, communities are pushing back against massive hyperscale data center projects, protesting against the noise pollution, environmental impacts, and blow to property values near these computer warehouses. In many places, elected officials are forging ahead to allow these multi-billion dollar deals in direct defiance of their constituents' wishes. Earlier this year, an Oklahoma farmer was arrested at a town hall meeting for speaking over his time against a 255-acre data center. Data centers are easy to build but much harder and slower to power, and their workarounds are egregious; For example, Elon Musk’s xAI Colossus 2 data center in Tennessee skips the local grid and runs its own natural gas turbines, worsening pollution and disproportionately impacting Black neighborhoods. But this tiny plot of American chestnut trees, just a little over an acre of land, has become a site of resistance in West Virginia. 

I first heard about the Clements chestnut grove after a 404 Media reader emailed me about it earlier this month, saying that the trees faced “potential destruction after November 1, 2026.” Social media posts called the threat “urgent,” and claimed bulldozers are lined up, with headlines declaring the grove is “about to be destroyed.” In the last few weeks, the plight of a grove of fragile chestnut trees trembling in the shadow of a data center has gone viral. Goldberry Grove, a local regenerative chestnut orchard, posted “They’re bulldozing the American chestnuts after November 1st,” sparking more outrage. 

In 2023, Governor Jim Justice announced a multi-billion dollar deal with Fidelis New Energy for a lifecycle carbon neutral hydrogen production facility, including the “Mountaineer GigaSystem” and the Monarch Compute Campus. Nscale acquired the American Intelligence & Power Corporation, sponsored by Fidelis New Energy and including the Monarch campus, in March. 

“The citizens of Mason County are elated with the announcement of Fidelis New Energy selecting Mason County, West Virginia, for its multibillion-dollar state of the art carbon neutral, clean hydrogen complex,” John Musgrave, the Executive Director of the Mason County Development Authority said in Justice’s 2023 press release about the new data center plans. 

With construction of the campus underway today, citizens do not seem elated. In September, heavy storms reportedly caused neighborhoods nearby to flood, as runoff from the construction site sent water and debris into adjacent properties as a section of the site’s fencing gave out. The Department of Environmental Protections (DEP) ordered Nscale “to immediately repair and strengthen the stormwater controls that failed, install additional erosion and sediment controls where needed, and take whatever actions are necessary to prevent similar releases from occurring in the future.”

Nscale told local news outlet WOWK 13 that it would pay for the damage. Separately, the company is trying to buy out homeowners, but the people living there are resistant. “What they’re saying is one thing, but they’ve got to prove it,” one homeowner told WOWK 13. “I’m staying. Fight for every one of these people.” 

The fate of the trees has been a conversation among conservationists in the area for years. The American Chestnut Foundation (TACF) raised the issue in its May 2024 monthly newsletter: “Several communications have been made with [West Virginia Secretary of Commerce James Bailey] and he is aware of the chestnut trees at Clements. He assured the WV chapter that the trees will remain and be taken care of by Fidelis.” 

The newsletter bulletin continued: “Mr. Bailey stated, ‘We value the chestnuts and will ensure that they are cared for. The state maintains ownership and control of the property, but if that ever changes, then the trees will be taken care of.’” The TACF noted that the future of the grove was “unclear.” Three years later, the picture is sharpening, as chestnut experts and the data center developers clash. 

Sixty-two years ago, the first seedlings shipped out from West Virginia’s newly-designated Clements State Tree Nursery. The nursery was once owned by Mark Twain’s grandparents. The West Virginia Division of Natural Resources bought it in 1961 and the Division of Forestry operated it for decades onward. The nursery sold seedlings of several varieties of trees and kept 15 to 20 chestnut trees that showed promising traits of blight resistance. In 2021, the Charleston Gazette-Mail reported that the DNR quietly ceased operating the nursery, citing funding issues. “It was never set up to make money, but it was also never set up to lose the kind of money it’s been losing in the past 10 years,” then-Division of Forestry Director Tom Cover told the Gazette-Mail at the time. 

Today, all of the American chestnut trees remaining at the former nursery site are infected with the blight fungus, Mark Double, a past president of the West Virginia Chapter of TACF told me. Many have died over the years, he said, but most of the trees still have sufficient resistance to have survived for 35 years, in spite of multiple infections. “The WV chapter of The American Chestnut Foundation has been harvesting nuts from this orchard for more than a decade,” Double told me over email. 

Nscale’s Monarch campus is one of two data center projects planned for the area. Project Tamarack, which is owned by Starwood Digital Ventures, is planned for 10 miles south of the Monarch campus. West Virginia Governor Patrick Morrisey announced the project early last month. Morrisey promises 1,000 temporary jobs but just 300 permanent positions as a result of the project. Research shows that data centers rarely live up to promises about bolstering local economies. “Data centers do not bring high-paying tech jobs to local communities because they operate as infrastructure projects rather than traditional jobcreating businesses,” University of Michigan researchers wrote in a 2025 brief. “Although the construction of data centers can create many jobs, those are short lived.”

Residents were stunned to hear that another data center was coming to their community, telling local news outlet WSAZ that they felt blindsided. Despite not being located on the former nursery’s site, the fate of the American chestnuts have forced Starwood to address it, too. 

I emailed Starwood on Oct. 8 asking if its campus threatened the trees. “I checked, and that grove isn’t on Starwood’s land,” a spokesperson told me. Sometime after Sept. 24, according to archives of the site, Project Tamarack added a line to its FAQ section about the grove: “The former Clements State Tree Nursery is a separate property and is not part of the Project Tamarack development site. Project Tamarack has no ownership, control, or management responsibility for the property. Questions regarding the chestnut grove, the former nursery, or the property’s management should be directed to the West Virginia Division of Forestry, which operated the facility prior to its closure in 2021.” 

Nscale also claims that the trees aren’t on land it owns, either — yet. That section of land is technically still under a “plan to purchase agreement.” 

“We understand the deep community connection to the former Clements State Tree Nursery site, and we look forward to working closely with the Morrisey Administration—including the Department of Agriculture and Division of Forestry—and The American Chestnut Foundation regarding the American Chestnut trees while responsibly developing the Monarch Compute Campus,” a spokesperson for Nscale told 404 Media. They continued: 

“We want to be clear — there was never a plan to bulldoze the nursery or cut down the trees. Nscale does not currently own the former Clements State Tree Nursery but we are under a plan for purchase agreement. Once we own the land, we are focused on how to best care for the trees which, according to a recent assessment by an arborist, suffer from blight and adverse health conditions. We have always planned to incorporate the American Chestnut trees into the Monarch Compute Campus. As construction progresses, we will work closely with professional arborists and experts to determine how best to preserve and integrate them, covering all costs.”

Nscale Senior Vice President Pete Hollis told West Virginia Public Broadcasting: “There’s never been a plan to and there will never be a plan to bulldoze or cut down the trees.” 

On October 10, Morrisey announced that the trees would eventually be moved, framing this as a way to appease everyone. “Under the preservation plan, the existing trees will be relocated to a more suitable environment within the property, potentially as part of an arboretum or integrated landscaping. That work is scheduled for late 2027, during the trees' dormant season, under professional arborist supervision,” the governor’s office wrote in a press release.

Amy Metheny, a forest pathologist for the West Virginia American Chestnut Foundation, told WCHS-TV that relocating the trees would inevitably kill them. “It’s my professional opinion that if the trees were moved they would die within two to five years,” she said.

I asked Morrisey’s office if they’re aware that experts say the trees will die if moved. A spokesperson for Morrisey’s office replied: 

“We are aware of the concerns about transplanting and have discussed them directly with a member of the West Virginia chapter of The American Chestnut Foundation. The West Virginia Department of Agriculture’s assessment found that the trees are expected to continue dying under current site conditions. It also found that relocating chestnut clumps with proper care may improve their health, although some losses are expected.

“Mature chestnut trees from the same original source as the Clements planting are growing in and producing nuts in Tyler County. Approximately 500 nuts from those trees are expected to be used for additional planting to support continued restoration efforts.

“The company has said relocation remains its plan and that it will cover the cost. The property is subject to a legally binding purchase option signed in 2023 by the previous administration. We are working with the company and conservation experts to help secure the best possible preservation outcome. Specific questions about those relocation plans should be directed to the company.”

Much of the information floating around about the future of the grove — including the claims of looming bulldozers, the timeline for the actual threat to the trees, the “not it” statements between both data center projects, and the governor’s promises — has been a game of rumors and conflicting information. A representative for Goldberry Grove declined to provide an on-the-record source for their claims of a November 1 deadline, or the bulldozer claims.

But the attention turned to an otherwise largely overlooked and tragically rare tree species has stirred up the kind of public action that data center developers around the country have had to contend with when they want to throw miles of humming warehouses up in communities. On October 20, the West Virginia DEP hosted an in-person public comment session about Monarch’s air quality analysis and permitting, with a virtual meeting to come on October 29.

The campus’s environmental impact could extend beyond the handful of trees at Clements, according to documents filed with the DEP. The Charleston Gazette-Mail reported, based on those documents: “Construction activities would result in permanent impacts to over 8,400 linear feet of streams and over 1.7 acres of wetlands on one permit and over 5,900 linear feet of streams and more than 26 acres of wetlands on the other, according to the applications.”

In addition to being overwhelmingly unpopular with the communities where they’re built, and despite the trillions of dollars dumped into their development, data center projects and the industry at large around the world are starting to flounder. Australia’s NextDC AI data center operator’s shares dropped 40 percent on Wednesday after the company cited power delays and local pushback. Last week, Microsoft pulled out of its deal with Nscale’s Essex-based campus, following reports that the project might not be able to receive a power supply until 2030. Local community opposition to data centers has spurred legislation that’s slowed or halted these projects around the country, with some, including Microsoft in Wisconsin, canceling projects outright following petitions and protests. 

A Florida police department claims that scammers used AI to clone a sheriff’s voice, with scammers increasingly using voice clones that trick people into thinking they’ve missed jury duty or have a warrant out for their arrest are increasing.

According to Florida news outlet News6, scammers cloned Osceola County Sheriff Steve Moser’s voice and left a voicemail with a resident.    

“They cloned his voice and was using Steve’s picture as well to say it was him making the phone call, and it was not,” Sheriff Chris Blackmon told News6.

Impersonating a police officer is a third-degree felony in Florida. The Osceola County sheriff’s office did not immediately respond to a request for comment.

This particular scam cloning cops is spreading across several states, with reports of jury duty and arrest warrant scams targeting people in Georgia and Pennsylvania in the last week. These other reports don’t mention AI voice cloning explicitly, but note that the scammers sometimes have similar voices and tones as local police officers.  

Voice cloning scams have been around for years, with scammers using commercially available AI voice impersonation tools to target the elderly, families, and businesses. Last year, 404 Media investigated real-time deepfake scams that incorporated voice and video to defraud people. In 2024, we reported on the ways scammers cloned celebrity voices and use AI videos in YouTube ads to scam people.

Scammers often impersonate company executives or leadership to trick employees into sending money, and voice cloning appears to be an increasing part of that. In February, scammers cloned a managing partner of a law firm with AI. Ultimately, a private bank ended up transferring $108 million.

This is Behind the Blog, where we share our behind-the-scenes thoughts about how a few of our top stories of the week came together. This week, we discuss a new pod setup, video games, and Elon's spending habits.

JASON: Exciting news: I am building a podcast studio / riso printing room / place to film videos in. I believe all of my colleagues and many people in my life have been laughing at me about this, but this also may just be the narrative I’ve created for myself to inspire me to prove the haters wrong. Since the beginning of 404 Media, I’ve worked out of my garage, but feel like I’ve been going a bit crazy sitting mostly in the same room since essentially the start of the pandemic. And so I found not a coworking space but a very affordable windowless box that I shall turn into a content palace. 

I’m just going to try it for a few months to see if it makes sense or if I should return to working from the couch, but I’m quite excited about building it out. I only got it a few days ago and so at the moment it’s largely a bunch of boxes and random furniture I no longer had space at my apartment for. It’s got a drop ceiling with fluorescent lighting and landlord special laminate floors, but I also can do whatever I want to it, so I will likely do some sort of design on the walls, throw a rug down, whatever. 

This article was produced in collaboration with Court Watch, an independent outlet that unearths overlooked court records. Subscribe to them here.

Police arrested a Chicago area high school maintenance worker who attempted to meet a child for sex after investigators posed as a 15-year-old boy using AI generated images.

Christopher Thomas Mertens started talking to who he thought was a child through a dating app in September. According to an affidavit written by the department’s FBI Task Force Officer McKailey Scott, the investigation was done by the Livonia Undercover Resistance to Exploitation Unit (LURE), which identifies and prosecutes child sex offenders. Scott set up an account on the queer-focused dating app Taimi with “an AI generated image of a teenage-appearing male with the name ‘Caleb,’ and no age listed on the profile.  

Scott wrote that an unknown user messaged “Caleb,” and they moved to Whatsapp. After a few minutes of talking, the man asked Caleb for their age, and when Scott — posing as Caleb — said he was 15, the man said “Im interested” and “Im hard now.” 

The investigator used a combination of tools, including checking the man’s phone number using Thomson Reuters CLEAR and running images of his face through Clearview AI to identify him as Mertens. For days, Caleb and Mertens exchanged sexually explicit text messages and Mertens made plans to drive to meet Caleb on his day off, believing Caleb’s (non-existent) parents weren’t home. Cops used Flock to look up and track Mertens’ car and see that he was driving to the meetup spot, but Mertens was also texting Caleb the entire time to let him know he was on the way and when he arrived. When he got out of the car, Livonia police took him into custody.

Two hours after being booked, Mertens told jail attendants that he wanted to speak to Scott. When Scott got to the jail, Mertens admitted everything, including that he works at a high school in Illinois. The complaint doesn’t mention which high school.

Investigators have been generating images of children for years. One of the earliest examples of these is Sweetie, a computer generated child developed by Dutch children’s rights organization Terre des Hommes in 2013 that the organization used to talk to men online. Today, police departments at the U.S. border use AI-generated online personas designed to interact with and monitor protestors and activists, according to documents obtained by 404 Media.

In August, a 24-year-old Connecticut man was arrested after soliciting an undercover police officer who used what was reported as “age regression technology” to appear as a 14 year old in videos. He was charged with enticing a minor and criminal attempt to commit risk of injury to a child. 

In 2025, a 45-year-old man was arrested after investigators used AI generated images of children to catch him distributing child sexual abuse material. He was recently sentenced to 12 years in federal prison.

And in 2024, the state of New Mexico sued Snapchat after investigators set up a fake account, including AI generated images of children, that the platform allegedly recommended to “dangerous accounts, including ones named ‘child.rape’ and ‘pedo_lover10,’ in addition to others that are even more explicit,” according to the New Mexico Department of Justice.

AI generated child sexual abuse material is a rapidly growing problem for law enforcement. The National Center for Missing and Exploited Children has reported a massive rise in generative AI child abuse material, with criminals using it to extort and entice children, and children using it to bully and harrass other children.

Livonia police did not respond to a request for comment.

Sorry for the late newsletter today, we’re still a little sleepy. To ease you into this afternoon, I have an interview with the guy who coined the term “smutstep,” and has become a sensation among a very specific section of dark romance BookTok — as well as a lot of haters. I thought he had some interesting things to say about the content creation game and not taking yourself too seriously while respecting your own craft and audience. Enjoy.

The Creator of ‘Smutstep’ Wants You to ‘Let Your Freak Flag Fly’

Last week, a video came across my feed of a man saying horny lines over a dubstep beat. “On your knees,” he snarled. As the beat dropped, he growled, “Mmm, now look at me.”

“Instagram reels just showed me a guy who makes 'smutstep' for the booktok crowd and it is the funniest thing I have ever watched,” the person reposting the video on X wrote. The replies are predictably harsh: “I think this is actually the worst thing I have ever heard.” “Who the FUCK is demanding this 😭” 

People demanding this is why it exists. DARQ DAGR, whose real name is James, is a dubstep musician and voice artist who’s been working in the voice acting world for years. But smutstep is what’s made his work go viral, in large part thanks to the people who hate it so much. Smutstep emerged as a reaction to BookTok, a subculture within TikTok built by book lovers and their various followings. Within BookTok, the romantasy book genre thrives, with series like The Fourth Wing and A Court of Thorns and Roses (ACOTAR) gaining massive readerships for their blend of elven high fantasy and dragon-based erotica. The romantasy genre is not a quaint fandom. It’s a billion-dollar industry.

Like so many online communities devoted to things a little horny and misunderstood, the wider world tends to scrutinize and discredit romance readers, especially when Fae fucking is involved. James, who wants people to know he does read the books too, saw a way into two niches that could compliment each other.

“I think what I like about the whole culture — the BookTok culture and dubstep culture — it's just letting your freak flag fly in a way,” he told me in a call. “It’s accepting and embracing what you're into within a safe and healthy space. I think that's a really beautiful thing.” 

Read the rest of this story here.

MORE FROM 404

Sentry mode? The “security protocols” on a Tesla stopped a Homeland Security Investigations agent from placing a tracking device on it, according to a warrant reviewed by 404 Media. On Halloween of 2025, an HSI Special Agent filed a more than 60 page application to place a tracking device on the Tesla of a suspected drug trafficker in Milwaukee. Agents had been watching alleged drug dealer Sam Stair — a local landlord they believed was using his rental properties as the front for a drug trafficking operation — for months when they asked a judge to let them install a tracker on his car. According to the warrant application, Stair went everywhere in a white Tesla and the agents wanted to see where it was when they couldn’t keep eyes on it.

FROM AROUND THE WEB

Ghosting much... SpaceX will testify before the New York City Council this week about AI safety risks. This is after Elon and co ditched a hearing on Oct. 5.  

Last week, a video came across my feed of a man saying horny lines over a dubstep beat. “On your knees,” he snarled. As the beat dropped, he growled, “Mmm, now look at me.” 

“Instagram reels just showed me a guy who makes 'smutstep' for the booktok crowd and it is the funniest thing I have ever watched,” the person reposting the video on X wrote. The replies are predictably harsh: “I think this is actually the worst thing I have ever heard.” “Who the FUCK is demanding this 😭” 

People demanding this is why it exists. DARQ DAGR, whose real name is James, is a dubstep musician and voice artist who’s been working in the music industry and voice acting world for years. But smutstep is what’s made his work go viral, in part thanks to the people who hate it so much – and the ones who demand more of it.

Smutstep emerged as a reaction to BookTok, a subculture within TikTok built by book lovers and their various followings. Within BookTok, the romantasy book genre thrives, with series like The Fourth Wing and A Court of Thorns and Roses (ACOTAR) gaining massive readerships for their blend of elven high fantasy and dragon-based erotica. The romantasy genre is not a quaint fandom. It’s a billion-dollar industry.

Like so many online communities devoted to things a little horny and misunderstood, the wider world tends to scrutinize and discredit romance readers, especially when Fae fucking is involved. James, who wants people to know he does read the books too, saw a way into two niches that could compliment each other.

“I think what I like about the whole culture — the BookTok culture and dubstep culture — it's just letting your freak flag fly in a way,” he told me in a call. “It’s accepting and embracing what you're into within a safe and healthy space. I think that's a really beautiful thing.” 

James and I talked about modern artistry in a world where interactivity and engagement rule, why he doesn’t take all of this too seriously, and the value of escapism in a time when everything else sucks. And yes, I ask him to do the growl.

This interview has been edited for length and clarity.

SAM: I will admit just off the bat, I'm not in the BookTok world personally.

DARQ DAGR: Uh-huh.

I have nothing against BookTok. Beautiful community. But I can't get into it. I've tried like, ACOTAR. I haven't found my romantasy, I suppose. So I might ask you some dumb questions, but hopefully you can also serve as a little bit of an explainer for those of us who are not super in that world. Tell me how you got into this?

Yeah. So the thing is, it wasn't like when I was a kid, I was like, when I grow up I wanna growl. You know? I've been a trained professional vocalist for a really long time. I've done metal and I've done theatre, I've done pop, I've done rock. The reason I'm in Nashville right now is because I spent eight years in a country band that was touring the world. I just kinda do everything there is to do with my voice. I've been a professional performer and vocalist for a really long time. My wife is really into the BookTok world. And so I was exposed to the community and the books through her. 

The first video that went viral for me was when I had just left my country band. And all I wanted was for the way I was presenting myself and expressing myself to be true to the things that I was into, which is dubstep. I had fallen in love with dubstep. I think it's the most beautiful community in the world. And then, I've always been a huge fantasy fan. I've always been really into Lord of the Rings and The Witcher and all that stuff. So getting into romantasy was not a big step for me, because I'm already a fan of fantasy. Making that video was me kind of saying, ‘I feel like a lot of these BookTok boys are getting the growl wrong.’ And then I did a growl that I did from back in my metal days. And it just blew up and it created this whole community of people who were really into the growl. But I was not the first person to do the growl. 

I didn't know that. 

There's another creator who was doing it long before me. His name's Michael Ayok. I can't take credit for being the first to BookTok growl, but I definitely get a lot of recognition for it. 

Obviously I wanted to be producing dubstep music, but my community all wanted me to growl, so it was like, how do I make this make sense? And so I just started growling in my dubstep and inadvertently created a subgenre of dubstep called smutstep. It's all community driven though. Honestly, even the name smutstep was a comment on one of my videos.

Somebody had requested that I growl ‘sit down’ and I did a video growling it that did really well. So I was like, well, let's just make a dubstep drop out of this. It's community driven, and it's fun. I'm glad that people are liking it. 

I think the biggest issue though is that... I'm having fun. I do not take this seriously. Again, it wasn't a dream of mine to grow up and growl at people. I took two very niche communities — the dark romance BookTok niche community, and the dubstep community — that are not designed to be exposed to general culture. So there's a lot of hate that comes with that and a lot of negativity. People react very strongly to what I do. And I think that that's as an artist, I think that's effective art. You want people to be moved to an extreme. You know? But you cannot control what that extreme is gonna be. 

There are people that treat me like Marilyn Manson in the nineties. His art is meant to be taken seriously. He is making a social commentary where he's putting a mirror up to society and showing how ugly it is. Whether you love that or you hate it, it's meant to be taken seriously. Smutstep is not meant to be taken seriously. If you enjoy it, that's awesome. You know what I mean? I'm having fun with it. I love the community I've created around it. We're all having fun. But it's not meant to be taken as seriously as so many of the haters take it.  

Yeah, it's broken containment from these specific niche communities, into the wider world. Which is the curse and blessing of social media, I guess. Something breaks out and becomes viral and then everybody's aware of it, even people who maybe are not the audience. 

I sent you the link to the video that's got a million views now on X, and all the comments and replies are really rude. So just be warned. People are like, ‘I'm never gonna unmute this. I would never unmute this to save my life,’ that kind of thing. And like you said, it’s not that serious. If it's not your thing, that's okay. Move on.

But what do you think about those comments? The ones that are like, ‘this is gigacringe.’

That's been a lot of what I deal with on other platforms. Facebook and TikTok are way better. On Instagram, l have my comments limited to people who follow me because it's just so much hate.  

People are crazy on Instagram. People will say the most sideways shit on Instagram.

Yeah, and that's that's the thing. Like obviously, as much as we're all just having fun, of course it does mentally take a big toll on you.

You said this is community driven. Were people waiting to have this sort of thing enter their already-existing BookTok community, or where do you think it comes from? Why are people resonating with this?

Men in BookTok is not a new thing. Making it dubstep is my brand. But for me, smutstep is like the KISS makeup. It's a gimmick. Everybody needs something that makes them stand out. So right now, smutstep is what makes me different, it's what makes me unique. 

Most of my content is based off of people making requests in the comments. I think that may be why it's taken off the way it has. Obviously it has to do with me, but it has so much to do with the people who are enjoying it. This is something that I think in a lot of ways is actually a much more modern take on entertainment. From its inception, it’s community demand, and giving people what they want within what I wanna do. I'm not doing something I don't wanna do. It's like, all right, you guys want this? Well, I'm gonna make it, but I'm gonna go the dark romance route and make it super aggressive, because I want to make dubstep out of it, and people are like, ‘That's fucking dope.’

Are you familiar with like the romantasy theme park videos?

No. No.

Oh my god. You would get a kick out of this, and probably your wife would also, but it's part of SeaWorld. You know how around Halloween, theme parks will do like horror nights, and characters come out and chase you? It's that, but it's BookTok boys.

I'll send you a link. But it has gotten a really big reaction from wider culture, where people are like, ‘this is so cringe and weird.’ The execution is really cringe and weird, but it is responding to something people wanted, which was: romantasy in a more interactive way. Is there anything that people are demanding from you more than other stuff? 

There's a book series called The Fourth Wing and there's a dragon in it called Tairn. Somebody said that Tairn should be Scottish, because his name is Tairneanach, which is a Scottish name. So I did a growl, but in the Scottish accent. 

I do another series with masked growls. I'm supposed to do one today, if the mask gets here in time. It’s a Scream mask 'cause we're in spooky season, and people have been wanting that for as long as I've been doing this.

I saw the motorcycle helmet one. People really were into that.

I think what I like about the whole culture — the BookTok culture and dubstep culture — it's just letting your freak flag fly in a way. It’s accepting and embracing what you're into within a safe and healthy space. I think that's a really beautiful thing. Maybe it's just as I get older, but I'm just so disillusioned with reality and I constantly want more escapism for myself, and want to give others more escapism as well. So that's kind of my goal as an artist is just escapism.

That’s a huge appeal of fantasy and romance in general. I'm also a big Lord of the Rings fan. That type of fantasy I can get with. I wonder if Lord of the Rings launched today, if we would be calling it romantasy. I guess there's not enough sex in it though.

The thing about romantasy is, there does need to be pretty pretty graphic scenes in it. But Lord of the Rings has none of that. I do read the romantic stuff too. I don't wanna be somebody who just does it for the views. I do read both.

Do you think people have picked up on the smutstep genre since you started? Are there new people doing it? 

I haven't noticed, but that doesn't mean someone's not out there trying and just hasn't had a video hit yet. And to be honest, I'm not surprised. I think that it's one of those things... It's human nature to have an aversion to something new that they don't understand. It’s still in the phase where I think it's more ‘what the fuck’ than something that is socially acceptable. Which again, it was never meant to be broadly socially acceptable. It was meant to be niche-acceptable, which it already is. I don't think there's a lot of people trying to do it because I think people are too afraid of having to deal with the same stuff I deal with.

I would be remiss in letting you go if I didn't make you do a growl. Would you be willing to offer us a growl? 

I can. It's not going to be that loud, because I don't have a mic, but I can do one. Ready?

[Growls]

Did you hear it? Did it pick it up?

It was like a growl in the distance. I'll practice mine. I'll do it on the podcast maybe and you can critique my growl. 

Good Monday morning! Today's big story is an investigation from Joseph about cops and an AI tool that takes data from sites like Airbnb and Zillow and identifies specific properties. The internet is full of photos of the interiors of people's homes, from real estate and rental listings to short-term stays, and that data is in the hands of law enforcement.

And in other news, a very interesting interview with the whistleblower subject of Sorkin's new movie, some big legislative impact, and what caught my eye over the weekend around the web. Let's get into it.

AI Company Lets Cops Pinpoint Specific Houses Based on a Single Interior Photo

An AI company now lets cops take a photo and nearly instantly figure out what specific home it was taken in, using only clues in the photo itself, and is clearly pulling data from sites like Zillow and Airbnb. 

The tool, called Raven, shows the increasing power of AI capabilities available to local law enforcement agencies. 404 Media previously reported that both the Los Angeles Police Department and Miami-Dade Sheriff’s Office (MDSO) have bought access to Raven, which was previously called GeoSpy. Now, Raven, made by a company called Graylark Technologies, has added features that let cops analyze the interior of homes.

In a post on LinkedIn earlier this month, Graylark CEO Daniel Heinen wrote the company was introducing “the ability to geolocate interior images across millions of short-term rental units worldwide.” An accompanying demo video shows someone taking a photo from X of a washer/dryer unit inside a building. They upload it to Raven, select “Find Property: Interior and exterior property geolocation,” and then choose a region to search.

After a couple of seconds, the map interface zooms in on an area in Charlotte, North Carolina. On the right hand side, the screen displays a list of results. The top one reads, “Renovated! Walk Uptown/located in 2 Hot Spots!” That result directly quotes the language on a specific Airbnb listing in Charlotte, and the photos in the Airbnb listing show the same property.

Read the rest of this story here.

MORE FROM 404 MEDIA 

Spoilers ahead for “The Social Reckoning.” Did you catch this week’s interview episode? Jason talked to former Facebook employee Frances Haugen, who became one of the most important tech whistleblowers in 2021 when she exposed how Facebook’s algorithm was pushing people toward hate groups, showed Instagram was making teen girls feel worse about themselves, and showed that the company knew about its detrimental impacts on society. Haugen’s story is the basis for Aaron Sorkin’s new movie “The Social Reckoning,” which follows her evolution from model employee to leaker to formal whistleblower. Jason and Frances discuss what the movie got right about journalist-source relationships and how AI is supercharging privacy concerns. 

Listen to the weekly podcast on Apple Podcasts, Spotify, or watch the convo on YouTube.

This is Behind the Blog, where we share our behind-the-scenes thoughts about how a few of our top stories of the week came together. This week, we discuss AI and spirituality.

JASON: I think we’ve generally carved out a pretty good lane for ourselves in the broader AI debate, writing about its current capabilities, its current harms, and the fact it was trained on stolen content without saying that it will never be able to do anything of use. We are often tinkering with AI so that we can write about it from an informed perspective, and so every once in a while, while I’m working on a story I will see something where it feels like AI could possibly be useful for some specific part of the reporting. 

Howdy, it's hump day and we're rolling along. Today we've got an update on a very strange case involving a man whose image was resurrected from the dead via AI and made to appear in front of a judge to announced "forgiveness" of his murderer. 404 Media contributor Matthew Gault had a long conversation with the man's sister, who wrote the words and generated the video, and she said that swaying the judge was the entire point.

THE BIG STORY

Her AI-Generated Video Swayed the Judge. The Court Said it Carried 'Undue Emotional Weight'

A man convicted of manslaughter in Arizona will be resentenced because an AI-generated video of his victim speaking from beyond the grave was ruled to have carried “undue emotional weight” as an impact statement. An appellate court in Arizona ruled that the manslaughter charge will remain, but the judge must reconsider the length of the man’s prison term because of AI.

In 2021, Gabriel Horcasitas shot and killed Christopher Pelkey during a road rage incident. A jury found him guilty of manslaughter. During Horcasitas’ sentencing hearing, Pelkey’s sister Stacey Wales played an AI-generated video of her brother as part of her victim’s impact statement. In the video, which she scripted with her own words that she made an AI generated likeness of Pelkey say, the AI avatar of Pelkey forgives his killer and said they “could have been friends” in another life. 

“I loved that video,” the judge said while handing down the maximum sentence of 10 and a half years.

Wales told 404 Media that her intent with the video was to sway the judge. “Isn’t that what you want human connection to be?” she said. “Do you ever come out of a movie and say: ‘Well, that was too emotional. That was too powerful. Human connection is powerful, and trying to convey that to somebody that's never known another person, this person on the earth, to convey their lifetime of 37 years down to a two-hour sentencing event is hard to do, and I think that victims and victims' families should be afforded every opportunity.”

Read the rest of this story here.

MORE FROM 404 MEDIA

A flock of new laws incoming. Several recently introduced pieces of legislation would, if passed, dramatically curb federal agencies’ access to Flock cameras, with the lawmakers referencing reporting by 404 Media in their explanations for why the legislation is needed. “At a time of growing concern about the unchecked power of artificial intelligence, Flock is eviscerating the very notion of privacy by installing tens of thousands of cameras in communities across America without their consent,” Bernie Sanders said.

This is Behind the Blog, where we share our behind-the-scenes thoughts about how a few of our top stories of the week came together. This week, we discuss inbox slop, Claude cults, and more.

JOSEPH: This is something I’ve been thinking about for a while. Jason and Emanuel covered iLands, the AI agents that email people offering to do work. My thing isn’t exactly the same, but similar.

For months, I’ve been noticing that many people email me ‘tips’ (I put in quotation marks because they’re sometimes not tips, but more, my Instagram account was banned, please help me) that are CLEARLY written by AI. I can tell this because the AI responses seem to keep using the same or similar words that it believes would apply to journalists, or are how journalists speak, but as a real, human journalist for more than ten years, I know that journalists don’t actually speak like that.

Student journalists meet the moment, as we’ve seen most recently this week. As the Cornell University sexual abuse case highlights the importance of student journalism, public records reveal how schools try to censor on-campus news outlets. Claire Woodcock talked to students and legal experts about how censorship from above impacts their work. “One of the biggest things we’ve certainly found with censors over the years is they want to be called everything but a censor,” one expert said. “What they’re doing is everything but censorship.” 

How Schools and Universities Try to Censor Student Journalists

Student journalists, their advisors, and legal advocates say colleagues and administrators are attempting to undermine student reporting on Jeffrey Epstein, Gaza, and other issues that public institutions would rather not have their reputations connected to.

Public records obtained by 404 Media show administrators taking meetings with parents and alumni when they take issue with student reporting and asking advisors to compromise their professional ethics. They also show escalating tensions between student newsroom staff and administrators, prompting more legal aid requests to advocacy groups. 

This comes as student journalists at The Cornell Daily Sun earned recognition for breaking the “Cornell Seven” story regarding the lawsuit filed in New York’s trial court last month by a Jane Doe accusing current and former Cornell University students of drugging and sexual abuse. The student journalists told The Cut this week that a vague university and campus police statement from nearly two years ago prompted more questions than answers. Such is the basis for most accountability journalism on college campuses. 

The cases we reviewed involved the phase of the reporting process that comes later, where a factual dispute the newsroom has defended or a complaint where the person doesn’t bother to explain what is allegedly wrong with the students’ reporting. 

Read the full story here.

MORE FROM 404 

Look at my lawyer dawg. A New Mexico attorney used ChatGPT to generate briefs that included completely made-up witnesses and testimony while representing a man accused of shooting his wife. When the judges caught him, the lawyer blamed it on his own “stupidity.” I'll say.

A lawyer used ChatGPT to generate briefs that included completely made-up witnesses and testimony while representing a man accused of shooting his wife. When he was caught by the judges, the lawyer blamed it on his own “stupidity.” 

“It's little comfort to know that my stupidity is what brings us together this afternoon,” New Mexico attorney Stephen D. Aarons said in a hearing last month before five judges. He “assumed” ChatGPT would generate a “bulletproof summary of proceedings.” Instead, ChatGPT generated false testimony from “wholly fabricated witnesses,” according to an order of direct contempt from the New Mexico Supreme Court. Aarons was representing a man who was found guilty of killing his wife earlier this year; Aarons said he hadn’t told his client directly that he was caught using ChatGPT on the case because he’s currently serving life in prison.

The false testimony included a nonexistent person named Danny Stanton saying he received threats, and another made-up person, Linda Stanton, saying her husband received threats. Other false testimony from more AI-generated people in Aarons’ brief included statements “regarding the shooter’s clothing and appearance,” according to the order.  

Reuters reported on the case earlier this month. "I am remorseful but hopeful that the disciplinary board takes into account it was an honest mistake," Aarons told Reuters. "It is a lesson learned for all professionals who rely upon this powerful but sometimes unstable technology."

Aarons’ reasoning for trusting ChatGPT, he said in the August hearing, was that he’d heard of doctors using AI for medical research. 

“So, Counsel, do you watch the news? Do you listen to the radio? Do you read anything about what's going on in the world?” one of the judges asked Aarons. “Because the problem with lawyers relying on AI hallucinations is an above-the-fold story every single day.”

The court documents were first spotted by Rob Freund on X.

Lawyers getting caught using AI and submitting errors to the court has been a big problem for years, and the judge is right: It’d be very hard to avoid hearing about this pattern of “stupidity” unless you were living under a rock. There has been case after case after case showing attorneys failing their clients by submitting error-riddled AI-generated documents in their filings, and are the definition of “you had one job:” in the case of AI-brained lawyers, they can’t seem to resist taking shortcuts regardless of how risky it is for their own careers and their clients lives.

Like in Aarons' case, we sometimes get to hear and see judges go off on attorneys for this behavior. In May, judges in the Supreme Court of the State of New York Appellate Division let several lawyers have it for more than 20 minutes about their AI use and laziness, and called their failures “striking, concerning, disappointing, and saddening.” And it’s not just lawyers anymore; court reporters are submitting AI-generated errors, too. 

“So, either you buried your head in the sand. And that's a choice to do that. An intentional choice to be uninformed,” the judge continued to Aarons. “Or, you took a gamble. And neither of those are consistent with the code of conduct. So I'm really struggling with your response, saying, ‘Well, I didn't understand. I didn't know about hallucinations.’ My 13-year-old nephew knows about hallucinations. My 75-year-old stepmother knows about hallucinations. So, either you made a decision not to be aware of what's happening in the world in using this tool appropriately, which is a willful choice. Or you decided to roll the dice.” 

At the end of the hearing, Aarons admits that he hasn’t told his client directly about this mess, saying he’d only talked to his family members so far, telling them there was “a problem with the brief.” He said his client is hard to reach because he speaks Spanish and is in prison.

“So you didn't say, ‘I relied on ChatGPT, and it submitted lies to the court, and the court is mad at me, and now there's a hearing.’ I'm guessing you didn't tell your client’s family members that,” a judge asked Aarons. 

“I didn't talk about ChatGPT. I just said there was a problem with a brief that I filed,” Aarons said.

According to the order, the judges found Aarons to be in direct contempt of court, referred him to the Disciplinary Board for further consideration, and barred from appearing before the court pending the outcome of the board’s investigation. They also threw him off the case in question and assigned a public defender to his former client. 

While he waits for the board to decide what to do with him, Aarons was sanctioned $5,000, which he must pay to the State Bar of New Mexico Client Protection Fund.

Abusive, sexually explicit deepfakes have only become easier to make and more widespread in the last nine years. Since we first covered deepfakes when they appeared in 2017, they’ve never been the stuff of dark web sites or shadowy networks. They're found on social media networks, through basic web searches, and on easily downloadable apps and accessible forums. 

The most prominent hosts of non-consensual, AI-generated imagery are supported by some of the biggest web infrastructure providers in the world, getting their hosting, email, advertising, and content management systems from these companies. A new study examines how providers like Cloudflare, Google, Proton, Namecheap, and WordPress help non-consensual imagery sites thrive. Despite years of knowledge about these wildly popular sites, big tech companies continue providing infrastructure services that prop up abuse imagery, even though their own terms of service often prohibit illegal and harmful content, the study claims. 

Hany Farid, professor in Computer Science at Dartmouth College, Sophie Nightingale, a senior lecturer in psychology at Lancaster University, and Lancaster’s Sarah Morgan, who acts as the project coordinator on Nightingale’s “Protecting Ordinary People from Deepfake Harms” fellowship published their paper, “The Backbone of Abuse: How Infrastructure Providers Enable the Proliferation of AI-Generated Non-Consensual Intimate Imagery” in Stanford’s peer-reviewed Journal of Online Trust and Safety on Wednesday. 

In a six-week period between February and March, they identified 400 URLs based on keyword searching and Google Alerts, then narrowed the sites down to 88 that were actively hosting non-consensual intimate imagery. Most of the content on these sites was of female celebrities, actresses, pop singers, K-pop idols, and women working in politics or activism. From there, they used open-source web-analysis tools like WHOIS to see what infrastructure providers the sites used.   

“Five players emerged as dominant infrastructure providers: Cloudflare, Google, Namecheap, WordPress, and Protonmail,” the researchers wrote in their paper. Cloudflare provided “the lion’s share of services,” they wrote, by providing website hosting, content delivery network, and domain-name server services, as well as analytic tools. Google provided SSL certificates and advertising space for the majority of the sites studied, while Namecheap was the dominant provider of domain registrar services, WordPress provided the majority of their content management system services, and Proton provided mail servers.

Each of these service providers forbid customers and users from using their services for illegal activity. Publishing or threatening to publish AI-generated non-consensual sexual imagery is a federal crime in the U.S. and elsewhere, and sexual abuse imagery in general is illegal in many countries. 

Cloudflare, Proton, and Namecheap did not respond to 404 Media’s requests for comment about these findings. 

A spokesperson for Google told 404 Media in a statement: “Without the specific domains from the report, we can’t investigate these claims. We have strict policies against non-consensual explicit content — including AI-generated imagery — across all our products. We make it easy for people to quickly remove this content on Search, continuously update our systems to limit its visibility, and we prohibit monetizing or promoting non-consensual and sexually explicit content."

The Google spokesperson added that its advertising platform prohibits monetizing or promoting non-consensual and sexually explicit content, and treats this content as egregious violations, blocking ads on the sites or suspending the advertiser accounts involved. They also noted people can request the removal of non-consensual explicit images from Search, and pointed to Google’s recently-updated removal request process. Google has also updated its ranking algorithms by promoting non-explicit content where possible, they said, and demotes sites in search results that have a high number of removals against them.

A WordPress spokesperson said WordPress is “open-source software, not a hosting provider,” and wrote in a statement: “WordPress.org does not host websites or provide hosting services to sites that use WordPress, and we do not have access to or control over content published on independently hosted WordPress sites.” They noted that WordPress’s software is distributed under the general public use license, allowing anyone to use it for any purpose. “We take the issue of non-consensual intimate imagery seriously, but describing WordPress as providing services or infrastructure to these sites conflates the software a site uses with the services that host and operate it,” the spokesperson said.

In response to WordPress’s statement, Farid noted that while WordPress.org is not a host, WordPress.com is a hosting service. “It's operated by Automattic, which Matt Mullenweg also runs, and it hosts millions of sites. Any NCII site on WordPress.com is squarely within scope,” he told 404 Media. “Automattic provides services to self-hosted sites. Jetpack and the WordPress.com CDN serve images from i0.wp.com/i1.wp.com for sites that enable it, meaning the intimate images themselves can be served from Automattic infrastructure even when the site is hosted elsewhere. That's infrastructure by any definition.”

Farid also noted that WordPress.org maintains “an ongoing service relationship with self-hosted sites,” he said, with core updates, plugin and theme distribution, and security patches coming from WordPress.org servers. Farid mentioned Automattic’s ongoing legal battle with WP Engine, during which, in 2024, Mullenweg announced WP Engine was blocked and then unblocked from accessing WordPress servers; this “showed the project can and will cut a specific operator off from those services,” he said.

“So ‘no access to or control over’ is not entirely accurate,” Farid said. 

These providers do draw their own moral and legal lines on what kinds of customers they’ll tolerate and when they cooperate with authorities. After initially refusing to do so, Cloudflare dropped hate speech and doxing site Kiwifarms from its protection services in 2022, and terrorist manifesto host 8chan in 2019. In 2018, Cloudflare banned sex work harm reduction social network Switter from using its services, citing anti-trafficking legislation FOSTA/SESTA, and in 2017, it Cloudflare stopped services to neo-Nazi site The Daily Stormer. In 2023, victims of the sex trafficking ring Girls Do Porn demanded Cloudflare stop providing services to sites hosting their abuse; Cloudflare claimed it doesn’t have “control over the content of websites using those services,” doesn’t have “the ability to alter or remove content on them,” and doesn’t “have knowledge of the people or entities who post any specific content to such websites.” 

404 Media previously reported Proton gave the Swiss government payment data related to a Stop Cop City Protonmail account, which in turn handed it to the FBI, and in 2024, Proton gave Spanish police information that identified a pseudonymous activist.

Aside from the infrastructure issue the researchers were studying, they found that 312 of the 400 sites they initially identified were unrelated to deepfakes, like gambling or random travel or cooking SEO-slop sites, but were using non-consensual imagery keywords as a way to rank higher in search results. This shows how incredibly popular the marketplace is for AI-generated abuse imagery, and how sites are able to manipulate Google search to appear higher in results if they use related terms. 

Morgan told me that while journalists can expose site administrators (like in the case of MrDeepfakes.com, whose owner was identified by a massive joint investigation by Bellingcat, the CBC and TjekDet), and legislation can act as a deterrent, the research group wanted to focus on infrastructure providers as the “distribution supply” for these sites. “Creators are going to create and people are going to demand. We can’t stop that. But we can call for providers to cut the distribution supply and stop enabling these sites. This content needs preventing from being shared in the first place. All parts of the ecosystem have to work together to play a part,” she said.

The researchers recommend in their paper that these providers stop supplying services to sites hosting non-consensual imagery. 

“We aren’t saying that infrastructure providers are knowingly facilitating this content — or that they are aware of what’s on every site that uses their provisions — but it’s in providers’ power to vastly improve their moderation and cease services to sites as soon as they are identified as hosting this content. And to check sites that are reported to them,” Morgan said. “A gold standard response in our eyes would be a commitment from infrastructure providers to working with NGOs and governments across the globe who are searching for solutions that will lead to verifying URLs and sharing these with participating platforms so they can be blocked.”   

This is Behind the Blog, where we share our behind-the-scenes thoughts about how a few of our top stories of the week came together. This week, we discuss some small changes, an AI song, and internet soup.

JOSEPH: We’re making a couple of changes that might sound boring to someone who doesn’t read or listen to our work, but I’m actually legit excited about them, and I think they will be better for all of you too. I’ll talk about one now and maybe the other next week.

When I first started writing about online cultures and subcultures almost 10 years ago, one of the first scholars I read on the subject of internet anthropology was Professor Sherry Turkle. Her earlier books, including The Second Self (published in 1984) and Life on the Screen (1995) became something like a model for my own writing and this kind of reporting— approaching the internet with an ethnographer's mindset and taking nonjudgemental documentation of how people are behaving and feeling in a given space, even if that space is digital. 

Through talking directly to people in their own environments, whether they were in MUDs and early virtual chat spaces or Second Life and massive role-playing games, her work revealed endlessly interesting firsthand perspectives on lifestyles that people still found subversive, marginal, and at times perverse. The people she interviewed often said that digital worlds offered support, companionship, sexual satisfaction, gender play and freedom that the quote-unquote "real" world did not. These were topics traditional academia wasn't yet touching: cyber-sex, intimate relationships unfolding in chatrooms, embodiment in online worlds that felt as real to their inhabitants as the world their physical selves sat in behind a keyboard.

What struck me about those books — and later, in Alone Together, Reclaiming Conversation, and The Empathy Diaries — was often how fundamental the human needs for connection and vulnerability are, despite the technology being used to fulfill those needs. 

Fast forward to today. Something has shifted. “In 1984, I called the computer a second self,” Turkle writes in her latest book, Artificial Intimacy: Who We Become When We Talk to Machines. “Computers provoked an explosion of talk about what was essential about being a person and about how new computational ideas might illuminate our thinking about the mind. With the digital twin, the metaphor is turned on its head. It’s the agent who is in charge and imagines a version of you. Now you’re the second self. So when you are given time to get ‘back to your life,’ it’s been diminished.”  

Professor Turkle is a licensed clinical psychologist, the founding director of MIT's Initiative on Technology and Self, and the Abby Rockefeller Mauzé Professor of the Social Studies of Science and Technology at MIT. Artificial Intimacy: Who We Become When We Talk to Machines comes out on September 29.

The technology today is wildly different than it was in the 80s, but as Professor Turkle explores in our conversation for the 404 Media podcast, the need for empathy, vulnerability, and to feel understood remains central to the human experience. With increasing reliance on conversational “companion” chatbots like ChatGPT and Replika, we risk misinterpreting machine programming as care, and risk misattributing human qualities like empathy and emotional intelligence onto unfeeling, incapable robots. The result, she argues, is a population forfeiting its empathy skills at an alarming rate, trading difficult feelings and conversations with humans for convenience and instantaneous feedback.

“When people turn to a chatbot, they don’t have to dread the fatigue of genuine give-and-take. And they don’t fear judgment or abandonment. But when we talk to chatbots, we lose the skills required to interact with humans, so chatbots start to feel like a better option. With an alternative, disappointment with people feels unnecessary,” Turkle writes.  

What happens when people stop caring that there’s not a human on the other line? We got into that on the podcast.

“It's a product whose fundamental directive is to keep your attention. And so it matters very much that we sort of become glommed onto it as though it were a person when it isn't,” Professor Turkle said in our conversation. “But more than that, I think that it's very important to ask, what is the end game of this?” She referred to the interview Mark Zuckerberg gave last year when he said Americans can only have three real friends, and intended to let AI fill in the demand for more.

“Now that to me is a very dark endgame,” she said. “And so people who say, ‘What the hell? I kind of like it.’ Well, let's just think about where we're moving towards. Because it's not just a little entertainment. It's a corporate project to get us involved in certain kinds of relationships that are going to make tons of money and to keep us at our screens all day talking to sycophantic, seemingly loving entities that don't know who we are and don’t care about us.”

In our wide-ranging conversation, we talk about how we got here, the concept of “deskilling” one’s own capacity for human connection and vulnerability, whether there’s a way back out of this mess for us, and what happened when she asked a chatbot to roleplay her own legacy (spoiler alert: nothing good).

Listen now wherever you get your podcasts, or watch on YouTube. 

This is Behind the Blog, where we share our behind-the-scenes thoughts about how a few of our top stories of the week came together. This week, we discuss just doing it, machines that eat the internet, and an eavesdropping watch.

EMANUEL: It is a foundational, defining feature of 404 Media, but I sometimes forget that when I’m having trouble with a story, I need to just do it. 

This is most of all true when it comes to reporting. I spent a couple of weeks looking into AI generated music on Spotify appearing under the names of real artists. It’s an issue I and many other publications have reported on in the past, but whenever I look into it I am shocked by how prevalent the issue is. If you talk to any musician or spend some time clicking around music platforms yourself, you’re going to quickly find hundreds of examples of this abuse of the digital music distribution system.