An academic gets a rise from "breaching experiments" in City of Heroes/City of Villains

(Update at bottom) This is interesting. A university professor has released a sociology study (which will soon be made into a book) using players in the MMORPG City of Heroes/City of Villains as research subjects (see video at the bottom of this article which shows gameplay). I saw it discussed on Hacker News, and read the article on nola.com that prompted the debate. After reading the 2008 research paper ("Play and Punishment: The sad and curious case of Twixt") I thought that I would give some analysis here on my blog, and add some of my own thoughts regarding academic ethics.

From a high-level examination, the study seems harmless. In it, Loyola professor David Myers exploited differences between the online game's official rules (mostly governing gameplay and competition) and unofficial rules (mostly player-created social norms) to study group behavior. Starting in late 2006, he used his player to build specific scenarios and observed the reactions of other participants.

But drilling down to his methodology, the study veers into uncomfortable territory. Myers didn't just play the game, he actually used his hero character "Twixt" to sabotage other people's enjoyment by violating the game's unofficial social norms. In other words, he took the role of an antisocial griefer to an extreme in order to drive social and emotional reactions, which he then used to support his thesis.

Here's how Myers describes his methodology:

These three sets of behaviors – rigidly competitive pvp tactics (e. g., droning), steadfastly uncooperative social play outside the game context (e. g., refusing to cooperate with zone farmers), and steadfastly uncooperative social play within the game context (e. g., playing solo and refusing team invitations) – marked Twixt’s play from the play of all others within RV.

Translation: He killed other players in situations that were allowed by the game's creators but frowned upon by the majority of real-life participants. For instance, "villains" and "heroes" aren't supposed to fraternize, but they do anyway. When "Twixt" happened upon these and other situations -- such as players building points by taking on easy missions against computer-generated enemies -- he would ruin them, often by "teleporting" players into unwinnable killzones. The other players would either die or have their social relations disrupted. Further, "Twixt" would rub it in by posting messages like:

"Yay, heroes. Go good team. Vills lose again."

The reaction was overwhelmingly negative. People used message boards and in-game chat to tell Twixt how much he was hated, what a terrible player he was, and worse. While Myers was technically following the rules of permitted gameplay, he was violating the rules of conduct to such a degree that even heroes were begging for villains to kill Twixt in the game.

There were also some extremely disturbing reactions to the experiment. Myers received at least one threat to kill him in real life, although it's not certain if the player who said this was being serious, as Myers' real identity was not known to other players at the time. This video interview with Myers discusses the incident:

Computer Game Threat

All in the name of science, right? Anyway, here is how Myers' summed up the implications of his experiments:

In real-world environments, "natural" laws governing social relationships, if they exist at all, are part of the same social system in which they operate and, for that reason, are difficult to isolate, measure, and confirm. In Twixt’s case, however, two unique sets of rules – one governing the game system, one governing the game society -- offered an opportunity to observe how social rules adapt to system rules (or, more speculatively, how social laws might reproduce natural laws.) And, the clearest answer, based on Twixt’s experience, is that they don’t. Rather, if game rules pose some threat to social order, these rules are simply ignored. And further, if some player -- like Twixt -- decides to explore those rules fully, then that player is shunned, silenced, and, if at all possible, expelled.

I had a couple of additional thoughts about Myers' study. When I was in grad school, we devoted some time during the required proseminar to academic ethics. As I recall, experiments in which subjects were not made aware they were taking part in an experiment were to be treated very cautiously. The example that was brought up in class was a 60s-era study that involved a researcher monitoring the activities of men having sex in a public restroom that was known as a pickup spot for anonymous rendezvous. Such a study could not happen now without raising red flags, owing to privacy and consent issues. While Myers' research did not infringe upon players' intimate lives, he apparently did not get their consent and he did not reveal that he was an academic conducting research.

There is also the emotional element to consider. He was manipulating people's feelings for the sake of an experiment. That seems questionable, despite the suggestion in the nola.com article that he was surprised people reacted the way they did.

I believe Myers anticipated he would draw an overwhelmingly negative emotional reaction before he even started his "breaching experiments." It's not just that people hate to lose. Myers has studied video games since the 1980s, and played City of Heroes/City of Villains for thousands of hours before he started his research. Surely he knew that killing other players in such a fashion went beyond griefing -- these virtual characters are representations of the people who play them, and the advanced characters have taken huge chunks of personal time to build. The article on nola.com notes that "Myers was stunned by the reaction, since he obeyed the game's rules." This seems a bit hollow. After all, he acknowledged that his methodology employed "steadfastly uncooperative" tactics -- what was he expecting, a medal? The social rules may not be explicit as the official game rules, but he must have expected negative consequences for violating them in such a manner, just as employing uncooperative behavior in a real-world situation would prompt negative or even angry reactions from others.

Update 7/7/2009: I found Professor Myers' blog, and a post in which he addresses the academic ethics involved in his research. First, he suggests that this is not technically an "experiment":

... this study is not really an experiment. I label it as a “breaching experiment” in reference to analogous methods of Garfinkel, but, in fact, neither his nor my methods are experimental in any truly scientific sense. This should be obvious in that experimental methods require some sort of control group and there was none in this case. Likewise, experimental methods are characterized by the manipulation of a treatment variable and, likewise, there was none in this case.

He goes on to claim that the lack of consent from the people in his research does not constitute an ethical breach:

The matter of informed consent in this case, because of the reasons above, is really more of a legal than an ethical issue. You will note that I reveal no real-life identities. Nor do I use player globals. The names of toons I do include in the paper are entirely immaterial to the paper’s content and could easily be changed – which I am quite willing to do, though I think it will make little practical difference to the players involved. Most of them, I think, would rather have the names in rather than out. None of them, after all, despite the claims of some, were “treated” or “manipulated” or “harmed” in any way. They were simply observed in how they played within the zone, similar, perhaps, to observing how people shop inside a shopping mall.

This is a false comparison. The players were not "simply observed," they were subjected to what he calls "steadfastly uncooperative" behavior. If he were to employ corresponding behaviors in a shopping mall, the result would almost certainly be verbal abuse from mall patrons and ejection from the property for harassment.

Finally, he refers to a Terra Nova discussion on informed consent, which is a great read. However, he did not follow the same rules of engagement as Terra Nova author Constance Steinkuehler, who says:

My general MO to date has been to keep the lines between my professional identity and my Lineage identity transparent to whoever is interested, treating in-game disclosure of information about my 'academic' life the same way I treat academic disclosure of details about my 'game' life, based on the notion that I am bound to both communities to be generally forthright about what I do.

Feel free to add your thoughts below about these issues.

A video of the gameplay in City of Heroes/City of Villains: