A Blizzard of Procedural Justice

In July of 2013, the game design director for Treyarch’s Call of Duty: Black Ops 2, tweeted what should have been some pretty innocuous stuff about changes to that game’s gunplay:

Certain bits of the Internet then promptly flipped out and showed its ugly underbelly while wallowing in a river of toxic replies to what seem like tiny changes. Among other things, people threatened murder and threw up all kinds of ethnic slurs.

This is an example of a reaction that is thankfully extreme. And yet many of us have seen other, milder examples of players reacting poorly or out of proportion to some change to their favorite game. Some downgrade to their favorite gun or their favorite class that the game designers see as a benign change that benefits the game but which some players see as unfair and unjust attacks. “You nerfed my class/gun/loadout/spell/build/whatever!” is a common enough complaint.

And yet, some companies manage to sidestep that kind of visceral reaction, or at least dampen it somewhat. What exactly do they do that’s different? To answer that, let’s first consider the psychology behind concepts like justice and fairness.

Such differing ideas of what constitute “fair” treatment given identical outcomes have long been in the interest of psychologists, particularly those studying justice in the workplace. The research started off in the 60s by examining what people considered fair pay and distribution of other rewards relative to inputs like work, time, and nice bottles of scotch1 Since then, though, the field has expanded to include the fairness of the process by which decisions are made, and several “procedural justice” rules to live by in order to create procedural justice have been discovered.

Based on that research,2 here’s one such possible list of five things that affect whether players see a decision to change a game as just and fair:

  • Give players a chance to voice concerns and participate in the decision-making
  • Be consistent in your application of the changes
  • Be transparent about why you decided what you did
  • Appear (and in fact be) free from bias toward or against one group of players
  • Offer recourses for bad decisions

Given this list, let’s look at how one game developer discusses changes to its game in a way that probably makes people think that their decisions are more fair. Let’s look, once again, at Blizzard and their competitive, online shooter Overwatch.

Specifically, watch this short video from November of 2016 where Game Director Jeff Kaplan discusses major changes to one of Overwatch’s heroes.

Okay. Let’s look at how this video and Blizzard’s general approach to Overwatch changes address each of the rules above.

Voice and Participation: Kaplan notes that the Symmetra update is in direct response to feedback that players have been giving them. He also notes that players can help them fine tune the changes by testing them on the public test realm (“the PTR”) which is a playtesting server where anyone who owns the game can jump in and participate in the test. At several points Kaplan refers to the updates and refinement of the game as something that his team does “with you guys” through the process of playtesting. He closes the video by asking players to voice additional opinions about the changes on the official forums.

Consistency: This one is a bit more subtle but it’s worth noting that the change is consistent across the board. It’s not exclusive to premium members or people who bought the more expensive Origins edition of the game. It’s not included with Competitive mode but not in Quick Play mode. And the change will apply to everyone when it comes. If Blizzard had introduced a new character in place of the Symmetra changes but made it DLC, then that might have also run afoul of this rule.

Transparency: Kaplan is exhaustively transparent about the reasons for the Symmetra changes. In fact, he spends the bulk of the video explaining the reasons for the changes based on data and feedback from players. It’s not just “We think these changes will make her better.” There is that, but Kaplan also takes the time to explain how they too frequently saw Symmetra as bound to certain situations like defending the first capture point on specific maps, and how her effectiveness dipped once the match progressed past that point. The changes to the character are then framed as solutions to this fundamental problem.

Avoiding Bias: This one is kind of tricky to discuss since this update is specific to one character and the people who like to play her. But when you step back and look at all the Developer Updates and read individual patch notes, it becomes clear that Blizzard takes the wide angle view on their game and considers how changes to one hero affect the entire game. They consider the impact that one change has on other heroes and they communicate that in their communications with players. To illustrate, Kaplan spends some time in the Symmetra video discussing how her new ability, Photon Barrier, can benefit other heroes. He also specifically invites non-Symmetra players to share feedback about how playing against the new changes feels on the PTR so that their feedback can be taken into account before the changes are made final.

Offering Remedies for Bad Decisions: Of the five rules, this is probably the least relevant to the topic of nerfing and other balance changes. In the world of work, offering remedies would include letting people appeal a decision to an oversight board or complain to a government agency like the Equal Employment Opportunity Commission here in the USA. But what Blizzard does here is promise to look at the win rate data collected from people playtesting the changes and make adjustments before they are introduced to the full game. And they have a history of publicly undoing or reducing changes if they don’t play out like everyone thought.

So there you have it: five reasons why I admire how the Overwatch team at Blizzard is handling changes to their game. Undoubtedly there are still players who vociferously object to these changes and claim that they are unfair. But I suspect that it’s many fewer than it would be if Kaplan and his team weren’t so up front and deliberate about how the explain the procedures they went through to arrive at their decisions. It takes time but it’s worth it.3

Footnotes:

1. Adams, J.S. (1965). Inequity in social exchange. Advanced Experimental Social Psychology, 62 335-343.
2. For an overview, see Madigan, Jamie and Macan, Therese. (2005). Improving Applicant Reactions by Altering Test Administration. Applied H.R.M. Research, 10(2), 73-88.
3. Now, that all said, Symmetra is now OP. Pls nerf.

2 thoughts on “A Blizzard of Procedural Justice

  1. Its funny but the timing of this article couldn’t be at a better time 🙂
    If you know MCOC from Kabam (a mobile Marvel fighting game) then at the beginning of March they released an update the make the game play really differently and the MCOC community made such a huge noise about it that Kabam suppose to release another update anytime soon.
    Maybe if Kabam read the article before making all those changes, the MCOC community response was easier 🙂

  2. “Offering Remedies for Bad Decisions”

    This is very much addressed: the changes went first to the PTR, which lets everyone try it out and give feedback. There have been many proposed changes that were modified after PTR testing. Most recently, the Ana nerfs were dramatically scaled back.

Leave a comment

This site uses Akismet to reduce spam. Learn how your comment data is processed.