New Research Project: Fanfiction
Notes from the geekiest part of the internet's underbelly
Today’s Song: Superfan
TLDR
I was an avid fanfiction reader for ten years, between 2005 and 2015. A few weeks ago I discovered that the index pages of major fanfiction archives (like Fanfiction.net and Archiveofourown.org) could be turned into data on the trajectories of individual fandoms. Ever the data glutton, I have responded accordingly, collecting a dataset that covers the full history of Fanfiction.net.
This is purely exploratory. I’ve got no idea what I’m going to find, but whatever it is, it will be nerdy. Read on for more about why and how I’m doing this.
Table of Contents
I. Metroid message boards and psychic serpents
II. So, why fanfiction?
III. How do you research fanfiction, anyway?
X. Today’s song (lyrics)
Metroid message boards and psychic serpents
How do you generate buzz for something that’s already legendary?
It’s a tough question. See, it’s easy to get people on board with popular things. For example, back when Halo was the big thing, my church youth group hosted all-nighters at an old theater. They hooked an Xbox up to the projector and students flocked to the theater to play 2v2, cinema style. We ate pizza, drank soda, and cheered epic head-shots in the name of Jesus.
That’s buzz.
Anyone can capitalize on buzz. Generating it, though—adding to the bottom line of the legend—is harder. If you’re tasked with generating buzz you have to figure out how to go big, or else you’re just riding buzz that already exists. That’s easy if you’re a big fish, like a Hollywood studio, turning a famous book into a film. But what if you’re a mid-level website trying to do something more substantial than your normal “yaaaaay!” article? How do you go big?
So, I like to think that when some middle manager at Nintendo Power Source, the AOL portal for Nintendo of America, announced a Super Metroid promotion, his staff greeted him with confused stares.
Super Metroid is legendary. It is one of the best games made for the Super Nintendo, and one half of the inspiration for the metroidvania genre. It created the formula that other legendary games like Hollow Knight riff on. So, how do you add buzz to something that fans will be eagerly paying tribute to for decades?
I suppose NPS couldn’t have known the game’s future. But they knew it was good, certainly good enough to stand by itself, without their help. Maybe that idea—that Super Metroid didn’t need them to sell it—shaped their response to the challenge? In any case, their answer was prescient, foreshadowing the changing nature of fandom itself. To generate buzz for Super Metroid, they decided the ideal strategy was…
…to let the fans generate the buzz themselves, using the NPS site as an amplifier.
The idea was simple: NPS staffers would run a quasi-RPG set in the Metroid universe on their message board. Staffers would “direct” the universe, laying out the narrative constraints and providing updates on large-scale events. Beyond that, fans would take over the story, writing whatever they wanted.
NPS called the project Metroid: Blood of the Chozo. Fans were invited to create a character which, per the rules, had to belong to one of five guilds—Bounty Hunter, Engineer, Merchant, Marshal, or Trog. Beyond that the fans could write just about anything they wanted alongside the hundreds of other fans writing stories for their own characters.
An adventure story featuring a bounty hunter chasing Samus Aran who got bogged down by endless shenanigans, always remaining one step behind her? Sure. A survival horror story about about a young engineer stranded in a derelict, infested outpost on a lost planet? Great! Authors collaborated and indulged, making everything up as they went. One of the more memorable characters, Mr. Quartz, was fabricated by a writer who took inspiration from a quartz watch on his desk. It was that kind of thing, made possible by the free-wheeling atmosphere of the nascent internet.
It was also a wild success. The fans wrote over 6,000 posts. In 1996, Nintendo Power magazine wrote a four-page review praising the project, which I stumbled across in Wal-Mart one day after school. That was my introduction to Fanfiction: letting fans tell their stories, forging them into a community. A buzz-amplifier. It was awesome.
Then I set the magazine down and didn’t think about fanfiction again for nine years.
Fanfiction started gaining traction on the internet shortly after the internet went mainstream. Fans created websites where they could archive stories for the fandoms that excited them, ranging from Buffy the Vampire Slayer to the Redwall novels. Eventually, fans created a central repository for the fanfiction genre—Fanfiction.net.
I rediscovered fanfiction in 2005 when a rogue fanfic from Fanfiction.net, Harry Potter and the Psychic Serpent, began making the rounds as a PDF on the internet, filling the anxious void in Potter fans hungry for Book Five.
If you weren’t there, it’s hard to appreciate how rabid Harry Potter fans were for Book Five. Rowling released the first four Harry Potter books on a tight schedule of one book per year, from 1997 to 2000. The fifth book, Harry Potter and the Order of the Phoenix, took three years. The agitation produced by the delay was intense—so intense that it created an international market for fakes claiming to be Book Five.1
In America, fans who were anxiously waiting for Order of the Phoenix turned to fanfiction to fill the void. Harry Potter and the Psychic Serpent was one of the most popular. Some rumors even suggested it was an early leak of Book Five, but a quick read of the first chapter was enough to dispel that lie. It was good, but it clearly wasn’t Rowling.
For many fans, though it was spectacular, and certainly enough to get them interested in fanfiction.2
Harry Potter and the Psychic Serpent still feels, to me, like the start of fanfiction. In retrospect, though, it was more of a side effect. Fanfiction, and especially Harry Potter fanfiction, had been growing for years. It was inevitable that it would eventually leak into the mainstream. And it grew even further from there picking up momentum until it was an unstoppable juggernaut.
To give you an idea of just how big it became, here’s a graph showing the cumulative number of Harry Potter fanfictions on Fanfiction.net between 2000 and 2024. I’ve used a vertical line to mark the release of Book Five.
So, Why Fanfiction?
Okay. So, there’s an easy answer to this question, which is to tell you why I picked fanfiction. That’s simple: I’m a nerd, and when I realized that I could collect a dataset about one of my old, nerdy pastimes, a small part of my nerdy, nerdy heart sang.
If you’re just looking for the TLDR, you can take that and move on to the next section. But there’s a second way to answer this question, which is to explain why you should consider it. I’ve got a couple reasons for you.
Fanfiction is an overlooked, valuable source for studying competitive creativity. Most people ignore it since they view it as an illegitimate and slightly shameful pastime. But the two main fanfiction sites are competitive markets where tens of thousands of people create permutations of major creative works to compete for the attention of readers. Done right, you can probably learn a lot from the winners of that competition. What themes animate readers? What authors do?3
It’s surprisingly relevant. For the most part, derivative works are suppressed by intellectual property law, but in the places where that suppression cracks, it turns out derivatives have surprising selling power. In India, for example, the derivative work Harry Potter in Calcutta, a pure fanfiction about Harry Potter visiting India, was selling very well before it was struck down in a copyright lawsuit. In Russia, the first book of the Tanya Grotter series was clearly a rip of Harry Potter and the Sorceror’s Stone. Some countries barred it, but Russia allowed it under a legal loophole allowing for creative parody. The series quickly diverged and has since sold millions of copies.
Also, this kind of research is just sort of what I do here. This newsletter is built on two core ideas—everything is deeper than you think, and everything connects to everything else. I believe, earnestly, that if you study the details that other people consider too frivolous to invest time in, you will quickly find that it’s more interesting than you thought and that it connects to important things much faster than you would predict. I want to show you that, here. Consider this a proof-of-concept.
That last bit merits a final note. I’ll talk more about the ways that fanfiction connects to other things as I discover them. But simply put, fanfiction provides a visible set of numbers that you can use to make educated guesses about hidden sets of numbers.
For example, fanfiction authors are hobbyists. Do you want to know how the pandemic affected other types of hobbyists, like sketch artists or garage bands? By looking at how Covid affected fanfic authors, you can make a more educated guess about other hobbyists who don’t produce number trails.
You have to be careful about this, of course, because fanfic authors differ from these other groups in ways that matter. If you lean too hard into the similarities and ignore the differences you could wind up making guesses that are more educated but less accurate. But the learning has to start somewhere—and if you stumble across a rich trove of numbers then it might as well start there.
How Do You Research Fanfiction, Anyway?
There are three ways to study Fanfiction data that I can think of off the top of my head. There are probably more, but these are the first-pass approaches that I think most social scientists would identify quickly:
Trope analysis. Or, read the stories and try to quantify themes and tropes. Often there are emergent themes in a fandom that authors can’t predict when writing the original work. In the Harry Potter fandom, for example, there was intense (and surprising) demand for Draco Malfoy to be a more morally complex, thoughtful character than Rowling made him. Rowling didn’t, treating the desire for a redeemable Draco as a girlish fantasy driven by attraction to Tom Felton. It’s evident from the stories written about him, however, that what many fans wanted was a world where children were more complex than their parents hatreds.
Text analysis. Or, quantify the features of text to make inferences about the emotional state of the writer. Psychological researchers have been doing this since the early work of James Pennebaker. Some findings from this line of research are wild. My favorite is that pronoun usage reflects one’s psychological state; for example, depression makes people more introspective and more likely to use I pronouns.4
Trajectory analysis. A final approach, and the main one that I plan to use, is to look at the growth of fandoms over time. I’m going to unpack this a bit, but the basic explanation is that, if you can track the number of fanfictions published in a given fandom across time, you can see the events that fandom responds to.
Let’s talk a bit about how that last one works. To start, here’s a screenshot of Fanfiction.net, which served as the main repository for online fanfiction for well over a decade (before being eclipsed by Archiveofourown.org).
Fanfictions are organized by category, with an entire page devoted to Books, another devoted to Anime/Manga, another devoted to TV Shows, and so on. Pick a category and you are immediately taken to a list of fandoms that looks like the one below.
Some categories have thousands of fandoms. At present, for example, the “Books” category (pictured above) lists 2,601 individual fandoms. The number of fanfictions posted in each fandom is listed in gray parentheses next to the name.
From a social sciences perspective this data, by itself, is middling. The most interesting thing about it is that the varying fandoms clearly arrange themselves into a Pareto-like distribution where a few winners are responsible for most of the fanfictions produced.5
Beyond interesting features like that, however, a “snapshot” like this is of limited use. To make something of this data, you need to stretch it. One way to do this might be to collect more data on each fandom. For example, if you paired these numbers with data on book sales, you could probably say something meaningful about the relationship between fanfiction and profit.
There’s another way you could stretch it, though. The data above is a snapshot of Fanfiction.net at a single point in time. What if you added more points in time?
As it turns out, you can do this quite easily, thanks to the Internet Archive’s Wayback Machine. Fanfiction.net and the pages for each of its subcategories have been indexed continually since 1999.
The indexing is intermittent. For some of the earlier years (like 2008) there’s no data at all. But if you take it all together you have a detailed record of the history of all of these fandoms and their growth. Suddenly you can say meaningful things about the start of a fandom, its growth and maturation, and (in many cases) its death.
Look at enough of the individual fandoms and you can probably start saying meaningful things about the nature of fandom itself, or at least the part of it that spills over into writing. For example, the question I asked earlier, about how Covid-19 affected fanfic writers? That has an answer. I’ll be writing an article on it.
So, studying the trajectories of these fandoms is as good a place to start as any. I’ve already collected all of the data I need from Fanfiction.net; it took about twelve hours of copy-pasting, spaced out over a couple weeks. Effectively, then, I now have trajectory data on something like twelve thousand fandoms, which will allow me to see how writers across America have engaged with every major pop culture phenomenon for the last twenty five years.
I look forward to seeing where this geeky side project goes.
Today’s Song: Superfan
Superfan
Lyrics by James Horton
Audio generated via Suno
[Chorus]
Hey you're a superfan, superfan,
Rockin it like Superman, superfan,
If you want it baby yes you can, superfan,
Got the world sittin' in your hands, superfan,
Hey, you're a superfan
[Verse 1]
Hey watch me rock it,
Fat wad o' bills in my pocket,
Gonna hit the town gonna lock it down
Want the whole world gonna get it
Wanna hit the sky, I'm gonna hit it,
Gonna miss it if you're sittin' around
Watch me I got this
[Chorus]
Baby I'm a superfan, superfan,
Gonna rock it like I'm Superman, superman,
I wanna take it and I think I can, superfan,
Got the world sittin' in my hands, superfan,
Yeah, I'm a superfan
[Verse 2]
Hey babe you a fighter
Hit those dreams, burn 'em brighter
Chase that star, chase it higher, alright
Want the world, you gonna get it,
Want that sky, you gonna hit it,
Gonna miss it if you stay sittin, that's right
Hey take it, you got this
[Chorus]
Baby you're a superfan, superfan,
Rockin it like Superman, superman,
If you want it baby yes you can, superfan,
Got the world sittin' in your hands, superfan,
Hey, you're a superfan
[Bridge]
Superfan, superfan,
Yeah baby you're a superfan, superfan
[Outro]
Yeah baby you're a Superfan, superfan,
Makin' magic like it's Neverland, superfan,
You wanna make that magic yes you can, superfan,
Got the world sittin' in your hands, superfan
Yeah, superfan
Footnotes
The section of the Wikipedia article that I have linked to contains an erroneous estimate of the number of fraudulent copies of Harry Potter circulating in China. The Wiki estimates 15 million but that is based on an incorrect reading of the source article. The article (linked here) actually states that there were an estimated 15 million fraudulent copies of a famous Chinese novel, Wolf Totem, in circulation. In spite of that error, however, it is probably reasonable to estimate that the sale of fraudulent Potter works in China is somewhere in the millions range, given the number of fraudulent works (Wolf Totem was just one book, after all, whereas the HP franchise produced several fraudulent books) and the high demand for them.
The Draco Trilogy (see Footnote 3) was a contemporary of Harry Potter and the Psychic Serpent.
There are only two authors that I know of who made the jump from fanfiction to the mainstream publishing industry, and they are wildly successful. Cassandra Clare has sold well over forty million copies of her fiction works since she originally authored The Draco Trilogy, and E.L. James highly addictive Twilight fanfiction, Master of the Universe, was so good that she recognized its sales potential and re-wrote it as an original work, releasing it as Fifty Shades of Grey, which… well… c’mon.
The point is that authors who can crawl to the top of the fanfiction heap and appeal to mainstream publishers probably have what it takes to make it big.
The in-text link (https://pubmed.ncbi.nlm.nih.gov/29504797/) refers to a recent article that demonstrates the robustness of this effect. The original research on pronoun usage and psychological state was pioneered by James Pennebaker, who has written an entire book on the topic for the public. You can find that book here: https://www.secretlifeofpronouns.com/ (Please note: I make no money off of referrals. The link is for your reference, not my profit).
I say “Pareto-like” because we normally associate Pareto distributions with the 80-20 principle, where 80 percent of the outputs (fanfictions) are produced by 20 percent of the categories (fandoms). In the case of fanfiction, though, the Harry Potter fandom has produced more fanfictions than the other 2600 fandoms in the book category combined.
