Yesterday a16z published an article in which NYU Professor of Design, AI and Media Theory Ruby Thelot argued that “enshittification,” or the concept that technology platforms get worse after they lock in users, isn’t real. In reality, Thelot says, people use social media more than ever and report that they are satisfied with it. 

Thelot argues that, enshittification, a term coined by Cory Doctrow which has become one of the stickiest ideas of our era to explain the current state of technology and capitalism, is a misdiagnosis of what he describes as “platform narcissism.” The idea is that people, namely older people like Doctrow and western geratric millennials who used to get a lot of value out of Facebook, selfishly think platforms should continue to cater to their needs even though they aged out of them and should be focused on raising their kids. The platforms are actually popular and well liked, but by younger people who are using Instagram and deriving value from it in a way older people can’t understand, or by people in other countries and contexts than their own. Additionally, Thelot argues, the theory of enshittification allows users who should log off an excuse to stay on platforms that it is fashionable to hate. They spend time there, but say they hate it and can’t leave because they are trapped. 

“Perhaps the reason why the platform seems worse is because your life has changed,” Thelot says. “Over the same period of time, your goals have changed, and they no longer fit with the platform’s new features. Why the hell would a fifty-five year-old be watching Reels? Go take care of your kids.”

The indisputable fact of Thelot’s argument is that various social media platforms have more users and are more profitable than ever. This data is readily available from reliable market surveys, polls, and quarterly earning reports from the companies themselves. It’s important to note that companies like Meta have a history of juking the numbers, and that despite this it recently reported its first ever decline of 20 million users, but Thelot’s point stands: Instagram, Facebook, and TikTok are massively popular. 

I will also cosign Thelot’s advice that people should log off. There are many reasons why some people can’t, present company included, but it’s never not good advice and even if you have to be on social media you can probably reduce your time scrolling. 

Thelot’s argument falls apart when he tries to prove people are actually satisfied with their time on these platforms. To prove this, he points to research from American Customer Satisfaction Index, a private company which surveys customers’ satisfaction with various products and services. In its ACSI Entertainment Study 2026 it surveyed 30,886 customers and found that overall satisfaction with social media rose by 1 percent to a score of 75 out of 100. If satisfaction with social media is rising instead of falling and doesn’t have a terrible score, it’s doing the opposite of enshittifying and people don’t actually hate it, he argues.

Kudos to Thelot for finding data that appears to contradict the human civilization-wide backlash to social media and nearly the only political issue that a deeply divided country can agree on, but this is cherry picking. 

Here are the type things customers are asked to evaluate with a score of 0-100 in the survey this rising satisfaction score for social media: “Quality of mobile app, reliability of mobile app (minimal down time, crashes, lags), loading speed and reliability, ease of navigation, ease of using the site on different devices, speed and reliability of video clips, variety of services and information, freshness of content, relevance of content, ability to use site without disruption, personal relevance of ads seen.”

Customers scored all these aspects of social media highly (between 68 and 84), and I agree with their evaluation. Instagram loads really fast. I can scroll videos on TikTok very quickly. I watched one YouTube Short about Warhammer 40,000 lore, and then was served more than a dozen other YouTube Shorts about Warhammer 40,000 lore that were both fresh and relevant. I do not contest any of this and I don’t think that is why both my friends in the DSA and my Trump supporting family members agree that Instagram is bad. 

If we wanted to find out why Americans have negative feelings about social media or why we think these platforms are enshittifying, we might ask questions like: Does using Instagram make you feel good? Do you think children younger than 13 should have unlimited access to TikTok? Has Facebook been a net positive for your family?

Let’s look at some other data. A recent survey from The Searchlight institute found that voters “overwhelmingly believe social media has a negative impact on society.” Last year, anEmerson College poll found that Americans have low confidence in information they get from social media and that social media platforms are most responsible for the creation of “fake news.” In 2021, the Cato Institute found that 75 percent of Americans don’t trust social media to make fair content moderation decisions. In August, a Reuters/Ipsos poll found that three in five Americans support stronger oversight of social media companies. 35 percent said social media stressed them out. 52 percent said they spend too much time on social media. A Pew poll in July found that 6 in 10 Americans supported a social media ban for kids under 16. 

These opinions, especially as they relate to the impact social media has on younger people, has nothing to do with how fast videos load on Instagram or how effective the ads are. The apps function well technically and the ads are making money, this is self-evident and the reason we have a problem. The reason people feel like social media is enshittifying is that these companies promised to connect people across the world but in reality have chased engagement, catering to basest the human urges to hold our attention. Kids especially feel socially pressured to use these apps because that’s what every other kid is doing and where they feel their community is. It is such a clearly awful and obvious result of the way these apps have enshittified that in August Meta agreed to pay $17 billion dollars in penalties and finally curb the most dangerous aspects of its platforms over claims it endangered children. 

People need to be on Linkedin to find jobs. Municipalities and news organizations share important updates on social media first. You might be pulled onto Facebook or WhatsApp against your will because your local school or community of parents congregate there. That doesn’t mean they like it. It is possible to hate something with your entire being and still participate in it.

There are Warhammer and cute cat videos, but every scroll is a spin of a slot machine that cycles between hate, fear, jealousy, disgust, and horniness. We all see it and no amount of pro big tech propaganda, funded by the same firm that invested in these companies is going to convince anyone otherwise. 

“Elena Vasquez and Marcus Chen have appeared as volcano experts, astronauts, thriller protagonists, podcast hosts, and academic co-authors across hundreds of independently produced AI-generated documents, never having lived,” a new preprint research paper from Samsung and the University of Warsaw said.

The paper identified a number of names that co-authored hundreds of AI-generated academic papers, articles, and books. The authors don’t actually exist, but are instead names that large language models repeatedly produce when tasked with generating experts in certain fields. 

The paper, titled “The Ghost Couple: Correlated LLM Name Priors and Their Haunting of the Web and Academic Publishing,” utilized a known phenomenon where certain LLMs will keep coming up with the same names in certain contexts. For example, In June, Sam wrote about how ChatGPT, Gemini, and Claude were likely to use the name Elias Thorne in fiction they generated, and that the character was often a lighthouse keeper. Similarly, users noticed that if they ask ChatGPT to generate a software developer, their name will often be Marcus Chen.

The researchers were able to show not only that LLMs often default to the same names, but that they produce “correlated character ensembles,” meaning some names were more likely to appear together. Other names that were consistently generated by AI models include Elena Amara Okafor from Claude, Aris Thorne and Lena Petrova from Gemini, and Elara Voss from ChatGPT. 

Earlier this month, I reported a story about Research Gold, a company that offered what it claimed was human medical research, but that was in fact entirely AI generated. The founder and lead methodologist for that company was named Elena Vasquez. Research Gold removed Elena Vasquez from its site after I published the story. 

Michał Brzozowski, the lead author of the paper, told me that searching for these names on Google turned up other instances of AI generated personalties. For example, following the killing of Alex Pretti at the hands of U.S. Border Patrol agents in January, a rumor spread on Facebook that he was fired from his nursing job for misconduct allegations. Snopes reported that the false statement was attributed to “executive director Dr. Elena Vasquez,” who does not exist. 

Brzozowski was able to search databases of academic papers for the names they knew LLMs often generated.

“On Zenodo, a CERN operated repository that mints real DataCite DOIs, we identify 1,655 ghost-authored records claiming nonexistent journals with fabricated publication dates.” A DOI, or a Digital Object Identifier, is a string of characters and numbers used to identify academic papers. The researchers saw that many of the papers authored by these AI names were backdated, meaning their publication dates were different from the date they were uploaded to Zenodo. 

Anyone with a free account can create a DOI on Zenodo, but the existence of AI generated papers with DOIs has impacts on other parts of the web and academic publishing. 

“These [AI generated papers] carry real DOIs harvestable by any scholarly aggregator; the infrastructure for large-scale scholarly record contamination is already in place. Ghost names additionally appear on ResearchGate, forming synthetic research groups with collaborators drawn from multiple model families, and are indexed without verification by Google Scholar and Semantic Scholar [...] The academic record is being quietly haunted.” 

Research Gate and Google Scholar are both aggregators of academic publishing that are likely to come up in search results. 

The researchers say that different LLMs and versions of those LLMs generate certain names so consistently, they think they can use them to determine the provenance of AI slop. For example, the name Elena Vasquez was particularly common in content generated by Claude Sonnet 4, so papers that list her as an author were likely generated by that LLM. However, Brzozowski said that this level of accuracy might not hold for long now that AI generated content with these names is flooding the internet and feeding back into all AI models that are scraping the internet for training data. 

The fact that AI academic publishing is struggling to deal with the load of AI generated content isn’t new. We’ve previously reported that scientific journals have published AI generated text, that AI is impacting the peer review process, and that the open-access repository for preprint academic research Arxiv will now ban authors for a year if they are caught submitting AI generated work. The potential upside of this research is that these names might allow us to detect this AI generated content more easily.

Last week we published a story that revealed an Amazon warehouse where the company scans and destroys thousands of books for AI training data. The following is an interview with one of the Amazon employees at this warehouse. We granted this employee anonymity because they were not authorized to speak to the press. 

The employee worked at Amazon’s VGT3 warehouse, which is housed in the same facility as LAS8, where Amazon operates its print-on-demand business. Both operations are part of a larger complex of Amazon facilities in Las Vegas, Nevada. 

We discovered that VGT3 was used to destructively scan books for AI training data by placing a tracking device in a shipment of rare books that a bookseller suspected was being acquired by an anonymous AI company. We saw the shipment travel across the country before finally landing at VGT3. Online, Amazon employees who worked at VGT3 described the work of receiving shipments of books, cutting the spines off of them, and scanning the loose pages. 

The Amazon employee I talked to described what that operation looks like from the warehouse floor. 

This interview has been edited for clarity and length. 

404 Media: What do you do when you get to work at VGT3?

Anonymous: We have to walk past everything to get to the back of the warehouse. That's where I first saw all the scanners, and I was really wondering what was going on. I remember I even asked someone what they were doing and they said they’re not sure even the people who work there know what they’re doing. Of course, once I got over there and started putting two and two together, I kind of understood. And you know, I think people who work there probably know what was going on over there, but probably just didn't want to talk about it. 

We also saw them cutting the spines off the books and everything. And after they scan it, they throw all the loose leaf papers into a big shuttle. [Editor’s note: Shuttles is how Amazon refers to gaylords, which are big open cardboard boxes that are often used to store books in bulk.] So they're all mixed together. There's no way to put the book back together or anything. Where I worked, primarily they would bring us boxes of books, or sometimes there's shuttles of books.

What kind of books?

All kinds of books. When we first started a lot of them were brand new. Some of them are used as well. Like you could tell, they were liquidated from a library or something like that. We were even getting boxes of stuff from London. There were a lot of books from the University of London. There was even like, I don't know what you would call them, but it was like kind of stapled together papers that said that they were presented to Parliament on the behalf of Her Royal Majesty the Queen. So they were like government documents. I don't know if they're public. Probably they're public, but it was just interesting that we found all that in there. 

What did you think the books were for?

That's what we were wondering, and we were kind of like, ‘this is a lot of random information.’ They have literally any kind of book you could think of. They have ones in different languages. We saw a lot of German and Russian books, and we actually got whole pallets of Japanese books. Some of these books are brand new and they're like still sealed. A lot of the Japanese ones were, but yeah, we do get the stuff from the libraries, and that's the stuff I think is more mixed because they're not all nice in boxes and everything like that. They're kind of just thrown into the shuttle. 

What was it like working there?

Our job was to unbox the books, and honestly, they seem like a mess over there. Like they're not really well managed or organized at all. Their process wasn't solid. Their process changed every day. But we were pretty much putting them in a tote [a small plastic stackable bin] so that they can go be scanned in and sorted, so that they can go over to the people to cut the bindings off. 

They take them off the truck. The dock's right there, and then they stage them, which is just putting them on standby. People pick up the books and bring them to our station and we stack up the totes until it’s a whole pallet, and then the people with the pallet jacks come and they take the whole pallet.

And I think the next area, I think they call it receiving. What I've heard they do there is that they're scanning all the barcodes pretty much to see if they need it and just weed out duplicates. Something else they had us do was—we call it the trash but we don't we don't know exactly what they do with them—but one day we were taking all the duplicates and all the ones they said, “oh, we don't need these,” and we were throwing those in different shuttles. They're like yeeted in there, like they're not nicely placed in there or anything. And they say they're returning them to the vendors, but I'm like, you know, ‘do the vendors go through it and see which ones are actually worth it?’ Because it's like a tall shuttle that's probably six feet tall, six or seven feet tall. They're just piled in there. It’s also right next to where they start cutting the spines, and so I think right after that, that's where it's going [the trash]. 

What does it look like when they cut the spines off the books?

It's a machine people operate. Each of these machines is just like a little workstation, and there'll be one person at each of these stations. It's really safe because it has a little cover and it has a little area where you slide the book into, remove your fingers from the area, and then you press a button, and it just comes down and slices it. And then you remove the books after the blade is gone. 

After they cut the spines they have these kind of library carts, kind of like how you would stack  books, but instead, it's a stacks of paper and little cardboard things to separate—I'm guessing— the different books.

I saw the pages being thrown in a shuttle after they were scanned. We were walking by and we went over to one of the shuttles and looked into it because we could tell they were throwing the old books in it or the cut books in it, and it's just a bunch of loose paper like just sheets thrown in there.

Did you see the scanners?

They probably have like at least 20 scanners, maybe 25. They look like machines that count cash, it flips through the pages really fast. It kind of has something like that they load it into, and I can see their screens, and on their screens, you can see the scans of the book, and it's going really fast. 

What do you think about Amazon doing this?

At first what they were telling people is that it was for Kindles, but I just didn't believe that. I instantly was like, I don't think that's how they do it because of publication rights and stuff, copyright and everything. Then I saw it was for AI. I don't like it only because I wish I could take these books. There's so much knowledge and so much stuff in them and some of them look like they might be rare, and I've heard that they order rare books, and that's why I say some of these are so obscure. I definitely don't like the idea that they can't be reused or anything like that afterwards. They're reducing the amount of available copies for other people.

I can clearly remember the first time I saw a deepfake. I was waiting for the train on my way to the Vice office in 2017 when I got a Twitter DM from someone who saw it posted to Reddit. It was a short .gif from the start of a porn video, showing a woman lying on a bed and talking to the camera, only the woman’s face had been edited to look like Gal Gadot.

The deepfake was shockingly realistic for the time, and as Sam’s reporting on it from that day on has shown, deepfakes have primarily been used to harass and degrade women. We were surprised by how advanced the technology seemed at the time and knew that the internet and the law wasn’t prepared to deal with it. The technology has advanced exponentially since then, and the internet and the law has still not caught up to it. But as concerning as deepfakes clearly were the moment we laid eyes on them, the problem was never that we thought Gal Gadot actually made an adult video. Deepfakes can look convincing, but they never fooled us that what we were seeing was real. 

But for the past few months I’ve been looking at a new and strange type of AI generated nonconsensual image on X that I think will fool a lot of people. These images, which I’ve started calling subtlefakes, are not as clearly offensive, but they are a lot more convincing. The concerning implication is that it’s not just the technology that is advancing now, but that the people who wield it maliciously are getting more sophisticated. Much like “cheap fakes,” which in 2019 Data & Society described as an edited piece of media that manipulates the truth without requiring AI or any other advanced technology, subtlefakes are another form of synthetic media that has taken over the internet since the emergence of deepfakes in 2017.

These more convincing images are not AI generated nudes of celebrities or face-swapped identities onto porn videos, but AI generated images of celebrities that are subtly edited to make them more revealing or provocative. We’ve written countless stories about AI generated nonconsensual media of celebrities and non celebrities. It’s a form of harassment that causes psychological and material damage to people, and they don’t have to be believable to do that. Even if the image looks photorealistic, it is often clearly fake for the simple reason that it’s unlikely that the biggest actor in the world would suddenly produce hardcore pornography that’s posted by a random account to X. 

The images I’ve been seeing lately are often only slightly altered. One type of image, for example, will edit a real photograph of a famous actor walking a red carpet, but edit it to make her dress more revealing or to make her butt or boobs bigger. Another type is entirely AI-generated post-workout selfies at the gym. 

The only example I feel totally comfortable sharing is one that was called out and shared by the target of one of these fakes, actor Xochitl Gomez. As a side-by-side she shared on Instagram shows, someone had taken a real photograph of her at a parking lot, looking over her shoulder and smiling at the camera, and used AI to make it look like she was bending over in one image and putting her hand on her butt in the other. Similarly, someone had taken an image of her on a red carpet and used AI to make it seem like she’s turning her back to the camera and sticking her tongue out. These images are not just photorealistic, they also don’t have the usual context clues we’ve come to expect from nonconsensual images. I think I am very good at spotting AI generated images because it’s a big part of what we do here, and I don’t think I could tell these were AI generated if I was just scrolling on X. 

These images are almost always posted by verified engagement farming accounts on X, which pays the people who operate them if their posts get enough impressions. X obviously doesn't care if people are doing this and a bunch of other stuff that’s as bad or worse. I’ve seen at least one account that shared a few of these subtle deepfakes, but when I scrolled down their post history I found nonconsensual fully nude and sexual images that they monetized via other platforms, so that’s another reason why X accounts might be doing this. There are many, many accounts like this and they are earning millions of views.

I think it’s interesting because it’s a new, and as far as I know, yet to be covered use of nonconsensual AI images. It’s also important and in the public’s interest to know it’s happening because people should know what kind of AI content is targeting them, especially if it’s at the expense of other people. It’s also possibly a warning sign for how people’s AI use is evolving from abusive but obviously fake, to more subtle, less offensive, but much harder to detect. 

The other reason these types of images are going to be harder to moderate is that it’s easier to produce them. There are plenty of services that let people easily AI generate nonconsensual nudes of anyone, but they almost always involve some friction. People have to go to a sketchy site, install a sketchy app, or pay a small fee. As we’ve reported many times in the past, more easily accessible AI generators from tech giants can be manipulated to produce nonconsensual images as well, but generally speaking, most companies are trying to prevent people from making those images with their tools. While it’s not impossible, it does require some effort to bypass those guardrails. 

It is not so hard to bypass those guardrails if the nonconsual images people are producing don’t include any nudity or sexual acts, assuming the AI generator is even design to prevent those type of images. Most mainstream AI generation tools at least try to prevent people from generating any type of nudity. Fewer of them try to prevent people from generating the likeness of a real person. I can’t say for certain where most of the subtlefakes I’ve seen come from, but I know that at least some of them were made with X’s own Grok because some images included the Grok watermark. X has repeatedly been caught enabling nonconsensual images and only attempting to do something about it after public outrage. 

X did not respond to a request for comment.

“Large social media platforms that are really struggling under the weight of this for the first time,” Hany Farid, an image forensics expert and cofounder of deepfake detection firm GetReal, told me. You can see my full interview with Farid here. “They've always had a problem, but this seems to be the first time where they're like, ‘Oh man, this is really bad.’ And I think it's because the volume and the sophistication of the fakes is nothing we've seen before.”

The primary targets of deepfakes always were and continue to be women, but Farid said other nefarious uses of the technology are also becoming harder to deal with.

“If you go back 10 years, it was pretty clumsy,” Farid said. “It's getting more and more sophisticated. They're evolving. Cyber criminals evolve and that's going to make everybody's job much much harder as the years go on.” 

Our reporting on deepfakes over the years has taught us that people make them because they feel entitled to women’s bodies, because they are intentionally trying to hurt them, or because it’s profitable. I’m sure all those reasons apply to subtlefakes, but it appears to me that the current state of X is pouring gasoline on that fire. People use subtlefakes as advertisements for accounts other platforms where they monetize nonconsensual images, but that is rare. It seems that the more logical scheme is to monetize pure engagement via X’s ads revenue sharing program, which pays X users for engagement. It’s not a lot of money, but it’s easy money. Since X is not interested in moderating this content, one person can also operate multiple accounts, posting dozens of times a day. 

In general, my X feed is mostly different types of engagement farming. Another type of subtlefake I’ve seen a lot of doesn’t require AI at all. For example, this post about OnlyFans model Sophie Rain claims that Drake paid her $2,000,000 to take her virginity, and includes two random images of Drake and a completely unrelated TMZ video of Rain. People click on the video to see if she said what the post claims, and the account gets the engagement before they realize it’s a lie. 

This kind of easy, stupid, but effective garbage is what X is rewarding, and these subtlefakes are just one concerning outcome.

I think we’re pretty good at telling the difference between an AI generated image and a real photograph, but when I need help I call Hany Farid. 

Farid is the cofounder of GetReal, a company that specializes in detecting deepfakes and other AI-generated images, and is the developer of PhotoDNA, a perceptual hashing algorithm that helps companies automatically detect and remove some of the worst images that exist online, and that is now being used by every serious internet platform in the world. 

We started talking to Hany regularly in 2017, when Sam first reported on deepfakes. As that technology evolved and changed how we perceive reality, so have our conversations. I wanted to talk to him today on the podcast so you could hear of those conversations, and so that Hany and I could zoom out, reflect on the past few years, and speculate about where we might be headed. 

404 Media is a journalist-founded company and needs your support. To subscribe, go to 404media.co. As well as bonus content every single week, subscribers get access to additional episodes where we respond to their best comments. Subscribers also get early access to our interview series. Gain access to that content at 404media.co.

Listen to the weekly podcast on Apple Podcasts, Spotify, or YouTube. 

Become a paid subscriber for early access to these interview episodes and to power our journalism. If you become a paid subscriber, check your inbox for an email from our podcast host Transistor for a link to the subscribers-only version! You can also add that subscribers feed to your podcast app of choice and never miss an episode that way. The email should also contain the subscribers-only unlisted YouTube link for the extended video version too. It will also be in the show notes in your podcast player.

Amazon is buying massive quantities of books, scanning them for AI training data, and destroying them in the process. 

A 404 Media investigation was able to reveal Amazon’s book buying operation, which hasn’t been previously reported, by placing a tracking device in a rare book we suspected would be acquired by an AI company for training data, and following it around the country to its final destination. 

That final destination was an Amazon warehouse in Las Vegas, Nevada. Amazon employees who work at this location say all they do is receive massive shipments of printed books which they then cut the bindings off in order to scan the books more quickly. The printed book is destroyed in the process. The logo of the Amazon team that works at this warehouse, called VGT3, is a dinosaur, brandishing its teeth and with a book in its hands.

CGTrader, an online marketplace for 3D assets, found that one in six models uploaded to the site these days is AI generated, but that AI generated assets account for only $1 out of $90 in revenue on the site. These numbers show that AI generated assets are quickly flooding the marketplace, but that most people are not interested in paying for them.

“Buyers are voting with their wallets, and AI-generated content is struggling to compete,” CGTrader said in a press release about its 2026 market trends reports. The company says the gap between the surging supply of AI generated 3D models and the middling demand points to “a gap that undercuts the assumption that AI-made content is repricing the market.”

“The upload numbers alone would suggest a takeover,” CGTrader said. “The revenue numbers say otherwise, and buyers refusing to pay for AI-generated models is saying something bigger than no thanks: it is a signal of how far they trust AI generation itself. Which leaves the question the industry has been avoiding: an AI model may be cheaper to produce, but what is it actually worth?”

Like the Unreal and Unity asset stores, CGTrader is a marketplace where people can buy more than two million 3D models to use in their videos, games, or 3D printing projects, and has been around since 2011. The report it published is based on marketplace data collected between June 2025 and May 2026 and surveys of buyers. 

Alexander Spivak, a 3D artist who sells his assets on CGTrader, told me that he’s not against AI, but that “as an artist, I haven't yet found a way to seamlessly collaborate with AI.

Because in creativity, the most important thing is the process of creation, which I enjoy.

The result is a completely different story. And I haven't yet been able to integrate AI into my workflow in a way that allows me to continue enjoying [the process.]”

Buyers told CGTrader that the reason they weren’t buying as many AI generated assets their quantity might suggest is simple: they’re not as good as human made 3D models. Buyers said that quality was the number one factor in choosing what they buy, even more than the price. This was especially important to buyers who wanted models for 3D printing, where a bad model could break or not print correctly. Only 4 percent of those buyers said AI “works well.” The survey also found that most people who bought AI generated assets weren’t satisfied with them. 20 percent of buyers found it not good enough, and 7 percent used it only with heavy editing. 5 percent said it worked well. 

CGTrader’s findings come months after the company announced it was partnering with Tencent on an AI powered 3D model creation workflow. CGTrader CEO Dalia Lašaitė told me that the workflow doesn’t just generate assets from scratch, but that designers can use it to refine topology and textures, segment parts, create variations and prepare models for production. 

“AI can accelerate the more mechanical parts of the process, while the designer retains creative direction and control over the final result,” she told me. “For that reason, we think of this work as AI-accelerated rather than simply AI-generated. Our goal isn't to increase the volume of AI-made assets on the marketplace. It's to give designers better tools to work faster and focus more of their time on creative work. Ultimately, buyers will choose the assets that best meet their needs. The distinction that matters most isn't whether an asset is AI- or human-generated, but whether it meets the required quality standard.”

AI generated content is creating a discoverability problem across the internet. On Instagram, porn sites, YouTube, and music streamers, human creators are being drowned out by a flood of AI generated slop. Lašaitė acknowledged that, despite the current discrepancy between how many AI generated models are uploaded to CGTrader and how many people are buying them, the increase in uploads alone could make it more difficult to find human artists.

“AI uploads are currently growing faster than AI purchases, which makes effective discovery and ranking increasingly important,” she said. “Our approach is to prioritize quality and performance signals rather than raw volume, including how an asset performs commercially, how buyers rate it and other indicators of quality.”

Research Gold, a site that advertises services for medical researchers, including drafting peer-review ready manuscripts, systemic reviews and meta-analyses, claims that it’s “100% human-written, never AI,” and lists a number of PhD reviewers and professional methodologists on staff that carry out this meticulous, difficult work.

The problem: The PhD reviewers Research Gold lists on its site are AI-generated and don’t exist. Other methodologists it lists are real, but are not aware their identity is being used by Research Gold. When I tried calling the company, an AI agent that refused to concede it was AI answered and kept trying to sell me Research Gold’s services. Email and chat communication with the company were also AI generated. 

“Protocol, search, screening, extraction, risk of bias, statistics, and a publish-ready manuscript formatted to your target journal or committee. Led by PhD methodologists with peer-reviewed publication records. PRISMA 2020 and Cochrane Handbook methodology. Authorship stays with you,” Research Gold’s site says. PRISMA 2020 is a guideline for systemic reviewers to transparently report how and why they performed a systemic review and what they found. Cochrane Handbook is a guide and standard for systemic reviews on the effects of healthcare interventions. 

Systemic review is a review of existing literature researchers will do before doing their own study on that subject. Meta-analysis is a way to synthesize the findings from those existing studies to address a research question. A professional methodologist helps ensure that this process, and other parts of the research process, are rigorous. 

Research Gold introduces “The Team” that does this work under its About page. They include Founder & Lead Methodologist Dr. Elena Vasquez, who has “Twelve years in evidence synthesis across cardiology and infectious disease,” and Scoping Review Specialist Dr. Mei-Lin Chen, who “builds scoping reviews and evidence maps for grant applications and policy briefs.” Vasquez, Chen, and the other six members of this team don’t exist. Searches for their names don’t return any online footprint that matches the description on the site or a history of publishing papers. Their profile pictures are also clearly AI generated. 

A different section of the site listed another group of methodologists with profile pictures that appeared real. Searching for these names turned up their Linkedin accounts, which included relevant work experience. All of them are or were freelance methodologists or academics. Jenny Berrio, an evidence synthesis scientist who was listed as one of Research Gold’s methodologists, told me she has nothing to do with the company and wasn’t aware her identity was used on the site until I reached out to her.   

“I do not work for Research Gold, and I never agreed to be listed as one of their methodologists. I have no relationship with this company,” Berrio told me. “They are using my name, photo, and bio without my permission. I'm in the process of documenting the site and will be sending them a formal takedown request.” 

All the profile pictures for the real methodologists listed on the site are identical to the profile images these people use in their real Linkedin profiles. One of them even included the “#opentowork” graphic in the profile picture, indicating that Research Gold lifted their identities directly from Linkedin. 

Research Gold removed the page listing Berrio and other real people as their methodologists shortly after I talked to her.

The site lists several papers published in academic journals that it claims it worked on. I reached out to the lead authors of those papers but did not hear back. 

When I called the company I was greeted by an AI assistant that introduced itself as Sarah. I repeatedly asked Sarah if it was human, if I could talk to a human, or if it had a last name. “Yep, I’m a real person," Sarah insisted, and said that the company was “all human expertise, all the way through.” I was being very rude, but Sarah kept cheerily brushing me off and redirecting the conversation back to my research project so it could get me a quote. 

Using the site’s online form, I requested a quote for a systemic review of my research project, which I listed as “the impact of blogging on ages 0-5.” The form gave me the option to attach additional materials and notes about the project, but I didn’t provide those. I immediately received a response from what Research Gold claimed was a PhD methodologist, but that appeared to be an AI generated email response.

“Thanks for sending this over. Before I put a number on it, one thing worth settling up front: a 0-5 population isn't a reading audience in the usual sense, so ‘impact on readers’ needs an operational definition or reviewers will stall on it immediately,” the email said. “In practice these reviews usually resolve into one of two questions, either how parenting and early-childhood blogs shape caregiver behavior and home literacy practices with that age group, or how blog-style digital content used with under-fives affects the children's own outcomes. Which of those two is the study you have in mind? Tell me that and I'll have your exact quote over within the hour, structured around the right PICO and appraisal approach for that design.”

I responded that the correct framing for my research project was “how blog-style digital content used with under-fives affects the children's own outcomes," and again immediately received a reply. 

“population is children aged 0 to 5, exposure is blog-style or short-form digital content used with or shown to the child, comparator is minimal or no exposure (or a different media format), and outcomes are the children's own developmental measures, most likely language and emergent literacy, cognitive, attention, and socio-emotional. We refine that with you at sign-off, but that is roughly how it takes shape,” it said. “For the full review at your flexible timeline the price is $1,900. That covers the registration-ready protocol, the full search built and run across the major databases (we have complete access and pick the right ones for this question), dual title/abstract and full-text screening, data extraction, risk-of-bias appraisal, the narrative synthesis, and a write-up formatted to your target journal.”

The email sent me to a portal where I could pay the $1,900.

Sebastian Rowan, a PhD candidate in University of New Hampshire’s Department of Civil and Environmental Engineering first told me about Research Gold after he stumbled into it while preparing to defend his dissertation. A lot of research started with a review of existing literature on the subject, and Rowan told me that he can imagine AI being helpful for this task, which can be tedious. 

“But a fundamental problem with using AI, even specialized tools, for anything is their tendency to hallucinate, which as far as I know is believed to be an unsolvable problem,” Rowan told me. “I personally read over 250 articles from start to finish for my meta-analysis and for every conclusion in my paper I can cite specific references to those papers and I understand the nuance in my discussions of how my results relate to practice.”

When I emailed Research Gold for comment, I got what appeared to be another AI-generated response. 

“Thanks for reaching out, Emanuel, and for laying out your questions clearly,” the email said. “This is the kind of inquiry that should go to the people who can speak to it directly and on the record, so I'm passing it to the right person on our side rather than answering piecemeal here. You'll hear back from them at this address. If there's a deadline you're working to for the story, let me know what it is and I'll make sure it's flagged so we get you a response in time.”

Research Gold did not send me a comment in time for publication.  

While we haven’t seen evidence that credible researchers are using Research Gold’s services, generative AI has already impacted academic publishing. Scientific journals have to filter through a flood of papers with AI-generated citations, and some AI generated papers are being published by academic journals. In 2024, I talked to a researcher who believed the peer-review process itself might be compromised by AI generated text. 

Microsoft has introduced new limits to how much its engineers can spend on AI tools at work and told employees that maximizing AI use internally is not the company’s goal. 

This makes Microsoft one of the last major companies to rein in its employees’ expensive AI use. Scaling back maximalist AI use, or what some companies have called “tokenmaxxing,” is a trend we’ve covered in recent months as the price for using AI has increased while not always delivering commensurate productivity gains. 

Substack contributors are rejecting the company’s decision to flag AI generated writing on the newsletter publishing platform, with several creators saying the new AI detection features are a “witch hunt.” Some Substack users who use AI to help them write or edit articles say the new policy unfairly discriminates against them, while other users who don’t use AI in their writing at all are worried that Substack will mistakenly flag their writing as being AI generated and tarnish their reputation. 

“I’m not going to apologize for using AI in the creation process. I wrote for 20 years without AI, I could do it again if I wanted to,” Mack Collier, who has a small Substack called Backstage Pass, wrote. “My output would fall, my posts would be less structured, and it would be more obvious that they needed a good editor. Using AI improves the overall quality of my writing. That’s why I use it.”

“These detectors are notoriously, wildly inaccurate,” Alice Lemee, a ghostwriter and digital writing coach, said in a video she made about Substack’s new AI detection features. “All it takes is one false accusation for a writer to have their reputation almost irreversibly tarnished.”

Substack announced the AI detection features on Tuesday in a post from CEO Christ Best, who said that “It’s getting harder to tell what’s real on the internet” and that “when content made by no one takes over parts of the internet that are supposed to be human, it pollutes the commons and makes it hard to discover and hear human voices.”

Best explained that Substack has partnered with the AI detection tool Pangram, which is now built in to Substack and allows any user to scan a piece of writing. Pangram then produces a percentage-based score determining how much of the writing was AI or human generated. (Disclosure: Pangram previously bought an ad on 404 Media). We’ve covered research from Pangram, or research that relies on its AI detector before, showing how AI generated writing is flooding every corner of the internet. 

As we’ve previously reported, AI detectors are not perfect, and Pangram itself is not immune to labeling human content as being AI, and labeling AI as human. Max Spero, the CEO of Pangram, recently told us that the company is constantly working on minimizing errors, and that it estimates its false positive rate at roughly one in 10,000.  

“We have now built the mirror image. Pangram is a real machine, trained on vast amounts of text to make a probabilistic guess, and we have pointed it at your writing to work out whether a person is hidden inside,” Sam Illingworth, a professor and the author of Slow AI, said on his Substack in a post titled “Substack’s AI Detector and the Return of the Witch Hunt.” “To decide if there is a human on the other end, Substack asks a machine.”

Substack also added a way for users to add a “How I make this” statement, where they can explain their writing process and disclose if they use AI. 

“We’re not against people using AI to assist their work, and we think people should be free to choose which tools they use to express themselves,” Best said. “Some on the platform are human-writing maximalists, others are publicly exploring the frontier of using AI tools while sweating the details to make work they stand behind. We use AI all the time at Substack to write software, do research, and build product features like clipping, translations, and more. But people should know what they’re getting.”

“I think the best way for Substack to address this is to invite further dialogue,” Illingworth told me. “I actually really like the new feature that lets authors tell their readers how they construct their newsletters using AI, and this is the kind of dialogue they should encourage rather than ascribing a score that we know to be incorrect. My worry is that not only do these AI detectors generate false flags for non-native English speakers and neurodiverse writers, but they also remove any opportunity for dialogue because they start from a position of suspicion. Whereas what we need to be doing is developing opportunities for trust between readers and authors.”

"The tools we've introduced today are intended to increase transparency and give readers more context, not to prohibit or penalize AI-assisted writing. They do not impact discovery on the platform,” a Substack spokesperson told me. “We're also encouraging Substack publishers to add a ‘How I make this’ statement, where they can explain their process directly, including how they use (or don't use) AI, and set expectations for readers. Creators can disable detection on their posts pre- and post-publication, as well as report and remove scans on their own work that they believe are mistaken. Find more information on how the features work here.”

Substack’s attempt to detect and label AI generated content was also celebrated by many readers who are exhausted by AI slop and other internet platforms that can’t or don’t want to label it. I just published story about how there is so much unlabeled AI slop on Spotify that people are now creating their own sites to detect and label it without the company’s help.  

But the backlash from some Substack users shows that companies can’t just flip a switch if they want to ban or give users the ability to filter out AI content from their feeds. As the flood of AI writing and images we see online and the real world every day makes clear, generating slop is easy. What we’re going to do about it is still an unsolved problem.