AI-Powered Animal Crossing Villagers Begin Organizing Against Tom Nook

A software engineer in Austin has hooked up Animal Crossing to an AI and breathed new and disturbing life into its villagers. Using a Large Language Model (LLM) trained on Animal Crossing scripts and an RSS reader, the anthropomorphic folk of the Nintendo classic spouted new dialogue, talked about current events, and actively plotted against Tom Nook’s predatory bell prices.
The Animal Crossing LLM is the work of Josh Fonseca, a software engineer in Austin, Texas who works at a small startup. Ars Technica first reported on the mod. His personal blog is full of small software projects like a task manager for the text editor VIM, a mobile app that helps rock climbers find partners, and the Animal Crossing AI. He also documented the project in a YouTube video.
Fonseca started playing around with AI in college and told 404 Media that he’d always wanted to work in the video game industry. “Turns out it’s a pretty hard industry to break into,” he said. He also graduated in 2020. “I’m sure you’ve heard, something big happened that year.” He took the first job he could find, but kept playing around with video games and AI and had previously injected an LLM into Stardew Valley.
Fonseca used a Dolphin emulator running the original Gamecube Animal Crossing on a MacBook to get the project working. According to his blog, an early challenge was just getting the AI and the game to communicate. “The solution came from a classic technique in game modding: Inter-Process Communication (IPC) via shared memory. The idea is to allocate a specific chunk of the GameCube's RAM to act as a ‘mailbox.’ My external Python script can write data directly into that memory address, and the game can read from it,” he said in the blog.
He told 404 Media that this was the most tedious part of the whole project. “The process of finding the memory address the dialogue actually lives at and getting it to scan to my MacBook, which has all these security features that really don’t want me to do that, and ending up writing to the memory took me forever,” he said. “The communication between the game and an external source was the biggest challenge for me.”
Once he got his code and the game talking, he ran into another problem. “Animal Crossing doesn't speak plain text. It speaks its own encoded language filled with control codes,” he said in his blog. “Think of it like HTML. Your browser doesn't just display words; it interprets tags like <b> to make text bold. Animal Crossing does the same. A special prefix byte, CHAR_CONTROL_CODE, tells the game engine, ‘The next byte isn't a character, it's a command!’”
But this was a solved problem. The Animal Crossing modding community long ago learned the secrets of the villager’s language, and Fonseca was able to build on their work. Once he understood the game’s dialogue systems, he built the AI brain. It took two LLM models, one to write the dialogue and another he called “The Director” that would add in pauses, emphasize words with color, and choose the facial animations for the characters. He used a fine-tuned version of Google’s Gemini for this and said it was the most consistent model he’d used.
To make it work, he fine-tuned the model, meaning he reduced its input training data to make it better at specific outputs. “You probably need a minimum of 50 to 100 really good examples in order to make it better,” he said.
Results for the experiment were mixed. Cookie, Scoot, and Cheri did indeed utter new phrases in keeping with their personality. Things got weird when Fonseca hooked up the game to an RSS reader so the villagers could talk about real world news. “If you watch the video, all the sources are heavily, politically, leaning in one direction,” he said. “I did use a Fox news feed, not for any other reason than I looked up ‘news RSS feeds’ and they were the first link and I didn’t really think it through. And then I started getting those results…I thought they would just present the news, not have leanings or opinions.”
“Trump’s gonna fight like heck to get rid of mail-in voting and machines!” Fitness obsessed duck Scoot said in the video. “I bet he’s got some serious stamina, like, all the way in to the finish line—zip, zoom!”
The pink dog Cookie was up on her Middle East news. “Oh my gosh, Josh :)! Did you see the news?! Gal Gadot is in Israel supporting the families! Arfer,” she said, uttering her trademark catchphrase after sharing the latest about Israel.
In the final part of the experiment, Fonseca enabled the villagers to gossip. “I gave them a tiny shared memory for gossip, who said what, to whom, and how they felt,” he said in the blog.The villagers almost instantly turned on Tom Nook, the Tanuki who runs the local stores and holds most of Animal Crossing's inhabitants in debt. “Everything’s going great in town, but sometimes I feel like Tom Nook is, like, taking all the bells!” Cookie said.
“Those of us with big dreams are being squashed by Tom Nook! We gotta take our town back!” Cheri the bear cub said.
“This place is starting to feel more like Nook’s prison, y’know?” Said Scoot.
Why do this to Animal Crossing? Why make Scoot and Cheri learn about Gal Gadot, Israel, and Trump?
“I’ve always liked nostalgic content,” Fonscesca said. His TikTok and YouTube algorithm is filled with liminal spaces and music from his childhood that’s detuned. He’s gotten into Hauntology, a philosophical idea that studies—among other things—promised futures that did not come to pass.
He sees projects like this as a way of linking the past and the future. “When I was a child I was like, ‘Games are gonna get better and better every year,’’ he said. “But after 20 years of playing games I’ve become a little jaded and I’m like, ‘oh there hasn’t really been that much innovation.’ So I really like the idea of mixing those old games with all the future technologies that I’m interested in. And I feel like I’m fulfilling those promised futures in a way.”
He knows that not everyone is a fan of AI. “A lot of people say that dialogue with AI just cannot be because of how much it sounds like AI,” he said. “And to some extent I think people are right. Most people can detect ChatGPT or Gemini language from a mile away. But I really think, if you fine tune it, I was surprised at just how good the results were.”
Animal Crossing’s dialogue is simple and that simplicity makes it a decent test case for AI video game marks, but Fonseca thinks he can do similar things with more complicated games. “There’s been a lot of discussion around how what I’m doing isn’t possible when there’s like, tasks or quests, because LLMs can’t properly guide you to that task without hallucinating. I think it might be more possible than people think,” he said. “So I would like to either try out my own very small game or take a game that has these kinds of quests and put together a demo of how that might be possible.”
He knows people balk at using AI to make video games, and art in general, but believes it’ll be a net benefit. “There will always be human writers and I absolutely want there to be human writers handling the core,” he said. “I would hope that AI is going to be a tool that doesn’t take away any of the best writers, but maybe helps them add more to their game that maybe wouldn’t have existed otherwise. I would hope that this just helps create more art in the world. I think I see the total art in the world increasing as a good thing…now I know some people would say that using AI ceases to make it art, but I’m also very deep in the programming aspect of it. What it takes to make these things is so incredible that it still feels like magic to me. Maybe on some level I’m still hypnotized by that.”