The Psychology of Game of the Year Debates

Ah, late December. The time when the gaming press gets its members together and tries to convince each other that one awesome game is more awesome than other awesome games –also known as the Game of the Year Awards. When I worked as part of the creative team on GameSpy.com we would lock ourselves in a conference room and argue literally for hours about the minutia surrounding every big title released that year in order to generate our awards. I’m also listening attentively to the GotY content over on Giant Bomb, which is dedicating a full week of multi-hour podcasts to the raw debates that generated its lists. 1

These podcasts are interesting to me because I keep seeing well established psychological phenomenon coming up, but almost as interesting is when a psychological quirk doesn’t manifest itself because the guys seem to be aware of its danger to the process and have taken steps to avoid it. So in this post I present my list of 2010′s Top 5 Biases That Affect 2010 Game of the Year Discussions. Sponsored by Crest Whitening Tooth Strips.2

#5: The Recency/Primacy Effect

The recency effect describes how it’s often easier for us to recall more information (and more salient information) about things that have happened more recently or items towards the end of a list. Similarly, the primacy effect means the same thing for items at the beginning of a list or that happened towards the beginning of an established time frame. Between the two of these effects, stuff in the middle tends to get forgotten or muddled.

Remember Bayonetta?

The impact on GotY lists should be apparent: If you’re studying a list of games released in the last year, it’s going to be easier to recall stuff about the first and last few games. We’re also more likely to recall details about games we played more recently (like Call fo Duty: Black Ops) or earlier in the year (like Bayonetta). Details and memories of games released toward the middle of the year (like Splinter Cell: Conviction) might not come to mind as easily.3

#4: Confirmation Bias

This is a big one for GotY discussions. Confirmation bias is our tendency to ignore or downplay information that dis-confirms our preconceived decisions or opinions and to pay more attention to and emphasize information that confirms them. If you go into a discussion of the Best Downloadable Game of 2010 thinking that Monday Night Combat should win, you’re less likely to think about its flaws (e.g., limited maps, repetitive comments from the announcer) and more likely to remember its strengths (e.g., class balance, fun character design) relative to someone who didn’t hold the same assumption. What’s more, you’ll probably say that the pros are more important to weighting your decision than the cons.

End of discussion! Wait, what?

Good ways to combat this are to get in the mindset of allowing people to challenge your assumptions and engaging in debate. It can also be helpful to list out the pros/cons (with help from others) so that you see them laid out and from a different perspective.

#3: Over-Emphasizing Salient Features

I wrote at length about this concept earlier, but here’s the quick version: When puny humans are asked to justify a decision, we tend to focus on the most salient or plausible explanations and then give them too much weight. To repeat my example from the previous article: if asked to explain why you favor Red Dead Redemption for the Best Action Game of the 2010, you may think about what should be included in the checklist for evaluating an action game, come up with “the weapons,” and then feel compelled to award or take away credit for how the game’s weapons feel and work. The problem is, the most salient and plausible factors may not be the ones that are really responsible for how much you enjoy the game. The weapons in Red Dead Redemption are largely unremarkable –the game’s appeal lies almost entirely in other areas and any weight given to how cool the weapons are is inappropriate at best.

This gun is irrelevant. Ignore this gun.

I keep seeing this come up in GotY discussions because professional game enthusiasts4 tend to hate using vague, worn out descriptors like “fun” or “awesome” or “polished” even though those words may be perfectly appropriate if a bit mundane. But these Internet auteurs are determined to have something more descriptive to say, so they cast about for something else and end up falling for the trap described above.

#2: Social Proof and Groupthink

This one is kind of a twofer since social proof and groupthink are separate but related. Again, I’ve written about social proof before, and the idea is that we will sometimes accept proclamations that are clearly at odds with our own senses just because we often have a desire to conform to the group’s standards. Soloman Asch showed this in a classic study where he got people to say that a long line was shorter than a short line simply by having someone planted in the group who would immediately pipe up and say so. The effect is even stronger with a group of strangers and statements with a less clearly defined correct answer, such as politics or game of the year awards. Which is why someone may not speak up when others in the group immediately jump on World of Warcraft: Cataclysm as the Best Role-Playing Game of the year, even though by most reasonable definitions it’s not a game.5

Deathwing thinks Line A is shorter. Are YOU going to argue?

The flipside is groupthink, which is when members of a cohesive, established group will ignore information, abstain from critical debate and accept otherwise questionable decisions in order to minimize conflict and maintain warm fuzzies. So again, Cataclysm might win, because so-and-so can be such a pedantic jackass about it and nobody wants to harsh the vibe or destroy the atmosphere of friendly discussion.

Interestingly, the Giant Bomb guys seemed to disarm these two biases from the start by joking about how they hate each other and how they anticipate rancorous arguments. This sets the stage that it’s okay –expected, even– to question each others’ decisions and engage in critical analysis.

#1: The Distinction Bias

Many GotY debates in categories like “Best [Genre] Game” come down to two similar contenders, resulting in protracted discussions where the merits of each candidate are obsessively scrutinized. This is a recipie for what’s known as the distinction bias. The idea comes from a theory that people engage in two modes of evaluation when pondering the merits of an experience: joint evaluation and single evaluation mode. The former is done when comparing multiple things at once and the latter when evaluating something individually.6

The distinction bias describes how when operating in joint evaluation mode we tend to over-emphasize and over weight otherwise slight differences between the subjects. If debating Need for Speed: Hot Pursuit and Grand Turismo 5 for Driving Game of the Year, we may make a bigger deal about Hot Pursuit’s lower frame rate than we would have if we were evaluating the game by itself. As a result, when operating in this comparison mode we tend to think worse of the loser than we would have if we had evaluated it without resorting to direct comparisons.

We need speed. And also unbounded rationality.

This is perhaps acceptable in GotY debates when we HAVE to pick a winner –it’s often the fine details that act as tie breakers. But the trouble may come when you have a mix of different types of games where two of them are similar. If you aim to trim the initial list to a set of three finalists, a tempting place to start is by comparing the most similar games (c.f., elimination by alternatives). Because of the distinction bias, the loser in that comparison may end up being evaluated worse than before and may end up getting cut from the list even though it was better than the non-similar games.

So there you have it. Five psychological phenomena that drive game of the year debates. Go listen to your favorite GotY podcast (again, I heartily recommend The Giant Bombcast) and see if you can catch them in action. If you do, post about it in the comments section!

Follow me on Twitter, Facebook, or RSS.

Footnotes:

1. Someone could write a full dissertation on Brad and Vinny’s debate over Minerva’s Den vs. Lair of the Shadow Broker as best DLC of 2010, by the way. Get on it.
2. Not really.
3. Which might actually help in this case, as Splinter Cell: Conviction was awful. You heard me.
4. God, what a weird phrase. But more accurate than “gaming journalists.”
5. Shut up. It’s an expansion pack.
6. You see joint evaluation mode all the time in game discussions, actually. It’s usually given away by comparisons of a game to some benchmark of the genre or platform. “Medal of Honor is not as good as Call of Duty“, for example, even though the discussion is ostensibly supposed to just be about Medal of Honor.

15 thoughts on “The Psychology of Game of the Year Debates

  1. Eh, maybe awful is a bit strong. I just got to a point where I was frustrated with all the shooty bits, which was not that game’s strengths. Also, the story was nonsensical at best. The sneaky parts were fun, though.

  2. Great article!
    I’m a games entousiast myself as well as a social psychologist (that is to say I earned my master’s degree…). I love seeing psychology put to good use by explaining certain game phenomena psychologically but even other peripheral game phenomena such as goty discussions (heated debates, I’m sure)

    What I was wondering though is how you’re expected to evaluate games whithout a frame of reference (other games). I would say it’s inherent in evaluating games because there really are no objective criteria by which you judge a game. For instance the frame rate can only be qualified as bad when compared to frame rates in other games. Without this comparison you would almost be inclined to accept the faulty nature of a game, because you simply have no other information on which to base your

  3. In reply to Erik, I think there are a variety of elements in a game that can be judged without the comparison approach. For frame rate you are really asking “Did the smoothness of the visual presentation impair/enhance your enjoyment of the game?” That can be answered without drawing in a comparison to other games.

    I think comparisons are often used by people to help articulate something they haven’t fully explained to themselves. One may say “I like GT more than NFS because it feels faster”. That’s not a far jump from “I like GT because (in part) the frame rate allows the game to be very responsive and visually impressive”. It is easier to say “this is better than that” and have the comparison take care of a lot of the details that would have to be extensively described in just evaluating the game alone.

  4. Interesting! The problem I’m most conscious of is your third point, Over-Emphasising Salient Features. Our responses to things are largely automatic (emotional? subconscious? pre-rational?) but we justify them rationally, even though we don’t fully understand our own mind. So how can we write a wholly truthful review without being vague? I know I enjoy playing King’s Bounty, but I don’t really know why, so when I talk about its delightful quirkiness and clever tactical design, I’m really just making an educated guess about why it’s good.

  5. In reply to Ryan,

    You do make a valid point, however I think that that kind of measurement will only be relevant up until the point you cannot make inferences from the game anymore without comparing it to others. Other, better, games set the standard against which one compares a game, especially when it comes to technical issues.

    I think the above goes for all tripple A games, so if you notice any drops in frame rate that are absent in other games it simply makes you wonder why the developers of this particular game weren’t able to develop a game without those kinds of hickups. Consequently you would possibly (and justifiably, in my opinion) rate that game lower than you would the other.

  6. @Erik and Ryan: I think some direct comparison is inevitable, but it’s a matter of degree. When you’re trying to decide if Game A or Game B is better, you end up doing much more direct comparison than using just one aspect of Game A and one from Game B as illustrations of things you’re trying to communicate about Game C in a review.

    Come to think of it, though, do you see such direct comparisons in critiques of other media or art? Do movie reviewers make direct comparisons to other movies when evaluating cinematography or special effects or costumes?

  7. @Fraser: I think you’ve hit on the reason I generally don’t read game reviews. I’d much rather listen to a podcast or read a message board and hear people cast about to describe their general experiences with a game. Following a strict format often forces reviewers to miss the point.

  8. Hmm, tough one.

    I admit that usually movie reviewers refrain from making elaborate comparisons with other movies in their reviews.

    But I don’t think it’s completely fair to compare these two art forms. Mainly because in movies the viewer is a passive spectator whereas in games the player is an active participant in unfolding the story.

    In a game the technological shortcomings are much more pervasive as they tend to be more structural than in a movie (AI, frame rate, controls, etc). When the world in which you act keeps throwing unconquerable challenges at you (crashes, frame rate drops, less than intuitive controls) the game will be less immersive where immersive “can be defined as “the sensation of being surrounded by a completely other reality […] that takes over all of our attention, our whole perceptual apparatus” (Lombard, M., and Ditton, T.). This would be detrimental to the overall game experience.

    I think this is what makes it nothing less than reasonable to look at other games when it comes to technological potential. Other games are the one source for determining if a game reaches the potential the technology can provide. Otherwise you’d have to expect a reviewer to be a black box everytime he reviews a game.

    By the way my reasoning concerns triple A games/movies only.

  9. Pingback: VVVVVV | Beeps & Boops

  10. Youre so cool! I dont suppose Ive read anything like this before. So good to seek out someone with some original ideas on this subject. realy thank you for starting this up. this website is something that’s needed on the net, someone with somewhat originality. helpful job for bringing something new to the internet!

  11. Pingback: VVVVVV | Beeps & Boops

  12. Pingback: Entry 03: The authority of team leader and the fail of democracy | My game design experience

Leave a comment