Posts Tagged ‘Starcraft II’
Closed Betas and Group Culture
A while back I wrote an article about the Attraction-Selection-Attrition model that I thought could explain why gamers choose what guild, clan, or message board community that they do. You can read the article for the details, but the gist of it is that people…
- Are attracted to organizations that share their values
- Are selected by organizational membership gatekeepers based on how well their values match the organizational culture
- Leave organizations over time as their values become (or are revealed to be) out of synch with the organizational culture
Founders, early members, and leaders have a disproportionate impact on defining values, which we call the “organizational culture.”
Recently we’ve been hearing an awful lot about two high profile closed game betas: StarCraft II and Halo Reach. It occurred to me that the ASA model of organizational choice could actually be applied to explain what kinds of people are attracted to closed betas and what lasting effects they have on player bases.

You have failed the secret group handshake!
First, people who like the game series and the subculture around it are attracted to the beta. These are probably going to be your most hardcore fans –people who gush enthusiasm for the game and everything that goes with it. Casual fans or non-fans are not likely to even be interested at this point without coaxing.
Second, those fans are willing to go through some pretty crazy hurdles to get selected for membership into that beta testing group. They’ll preorder your nutso collector’s edition. They’ll subscribe to services they don’t want just to get into the beta, or they’ll buy Halo ODST as much for the Reach beta code as the game. And we all heard those stories about early StarCraft II beta keys going for hundreds of dollars or more on places like eBay.
All along the way, the beta testers are defining the culture for the group by forming explicit or unstated but understood agreements about what kind of behavior is allowed, encouraged, or unwanted. If everyone in the Halo Reach beta is foul-mouthed and hyper-competitive how much of that do you think is due to those shared expectations formed by early adopters eager to get selected into those ranks?1
The third piece of the model is admittedly a little sketchy, as the beta group doesn’t really experience attrition of these fans in the same way that employers have employee turnover. If a beta opens up to the public or a game goes retail the hardcore beta tester groups aren’t likely to leave2 but there MAY very well be tension between the old beta testers and the new “scrubs” that flood the game. And beta testers may try to create their own sub-groups and isolate themselves in their own sub-culture.
Anyone have personal experience with this kind of thing?
Zerg Rushed by a Tiger? Just give up.
Neuroscientist and avid blogger Jonah Lehrer recently published a great article in the Wall Street Journal about what he and others call “the superstar effect.” The piece is well timed, seeing as it deals largely with the effect that someone like Tiger Woods has on his competition and Mr. Woods has in fact just returned to harass his competitors for the title of “Most Badass Dude Ever at Golf.” Lehrer describes the work of economist Jennifer Brown, who meticulously studied not just Tiger’s performance in high stakes golf games, but the performance of his peers:
Ms. Brown discovered the superstar effect by analyzing data from every player in every PGA Tour event from 1999 to 2006. She chose golf for several reasons, from the lack of “confounding team dynamics” to the immaculate statistics kept by the PGA. Most important, however, was the presence of Mr. Woods, who has dominated his sport in a way few others have.
…
Such domination appears to be deeply intimidating. Whenever Mr. Woods entered a tournament, every other golfer took, on average, 0.8 more strokes. This effect was even observable in the first round, with the presence of Mr. Woods leading to an additional 0.3 strokes among all golfers over the initial 18 holes. While this might sound like an insignificant difference, the average margin between first and second place in PGA Tour events is frequently just a single stroke. Interestingly, the superstar effect also varied depending on the player’s position on the leaderboard, with players closer to the lead showing a greater drop-off in performance. Based on this data, Ms. Brown calculated that the “superstar effect” boosted Mr. Woods’s PGA earnings by nearly $5 million.
The reason, Lehrer goes on to explain, is that when faced with such an overwhelming favorite in the odds, people tend to short sell themselves and not give their best performance, as if the outcome is predetermined. And what’s worse is that this need not even take place in our conscious thought to have an effect. And what’s worse than that is the fact that the phenomenon seems to be most potent with more experienced players. Veteran golfers play a good chunk of their game on autopilot, not wasting mental energy over analyzing every tiny movement, angle, or twitch. But when Tiger Woods is on the fairway, they may begin to overthink their strokes, their choices, and their plan –to engage in too much of what psychologists call “action identification.” The result is that they change the way they play and play worse as a result because they’re wasting their finite concentration on things that didn’t need it yesterday. Writer Malcom Gladwell of The Tipping Point and Blink fame also has a nifty article about this phenomenon, which you can read here.
When we talk about someone “psyching out” the competition, this is what we mean, and it appears to jive with actual scientific research. The WSJ article goes on to discuss how this same phenomenon happens in other competitive environments outside of golf, such as law firms or the executive boardrooms of General Electric, and how it’s especially potent in “winner take all” situations.
…Like, say a game of StarCraft! In the realm of video games, what this all made me think of is the importance of proper matchmaking based on skills and how some games seem to do it a lot better than others. Whenever I jump in to competitive game of Modern Warfare 2, for example, I can’t seem to take four steps without getting owned because everyone else in that game seems to be SO MUCH BETTER than I am. I think many of us have been in a poorly mached game where we round a corner to face the person dominating the top spot on a scoreboard and we just sort of sigh and wait to get headshot rather than try and fight back, especially if we’re squatting at the bottom of the rankings. Halo 3, on the other hand, always seems to group me with people closer to my skill level, and I have a lot more fun and win a lot more matches as a result.

A list of the people who would crush me in any given game of StarCraft II.
The superstar phenomenon is something that Blizzard seems to be actively trying to avoid in its ranking system for StarCraft II with its bronze, silver, and whatever levels of play and the ability to see the ranking of your opponent. Though not perfect and obviously still being tweaked, the system seems to go to great lengths to match players with opponents of similar skill. So I can be relatively sure that I’m not going to waste time second guessing my build order or metagame because I was matched against Tiger Woods, who in the context of this game would be some Korean dude who has been playing StarCraft for 12 hours a day for the last 10 years.
Regression to the Mean and Owning Some Chumps
Let’s say that you’ve hired a coach to help you improve your Slayer game in Halo 3. I’ve heard of stranger things. Let’s say this coach looks like Mr. Miyagi but he curses WAY more. He uses a variety of training and motivational techniques, ranging from grenade throwing drills to trigger finger sprints, doing everything he can to drive you towards perfection. You notice, though, that he eventually stops praising you whenever you rank at the top of a match. He did at first, but now when you earn more than your usual number of kills your coach stands stoically by, straight faced and not giving you a single word of praise for those outstanding rounds.
Eventually, you ask him why he never praises you when you do a really good job.
“Because,” he says, “I’ve noticed that praise doesn’t work. Every time I praised you for a really good round, your next round is always mediocre. And what’s more, when I yell at you for playing poorly, next round you always do better. Praising not only doesn’t make you better, it makes you worse.”
You pause for a second, then cry “You’re not my real dad!” and run out of the room, bawling like a child. Yes, you do. That emotional outburst aside, though, is your coach’s logic sound, given that you DO in fact perform worse every time he praises you for doing well and perform better whenever he rebukes you for doing poorly? Praise makes you do worse and berating makes you better, right?
Nope. Your performance following stellar rounds of Halo or Starcraft II any other game involving skill can be best explained not by the effects of praise or punishment, but by something called “regression to the mean.”
Let’s assume that if we looked at your performance over a bunch of matches and plotted them out with ending scores along the X axis and how often you end a match with that particular score on the Y axis. They’d probably form something close to a normal, bell-shaped distribution like this:

Figure 1: Your sick skilz, plotted
If we were to pick any single match at random, it’s more likely that your performance would be about in the middle somewhere –somewhere near the “mean,” which is basically another word for “average.”1 In this example, that’s 10 frags. It’s rare that you’re at the very top (17 frags) or bottom (3 frags). In fact, if your performance follows a normal distribution like the one above, then the following will be true:
- 68% of your matches will end with scores between 8 and 12
- 95% will be between 6 and 14
- Only 0.6% will be under 4 or over 162
And even if your distribution is a little skewed because you do well more often than you do poorly, the numbers won’t change much until things get REALLY skewed. At which point no amount of coaching is going to change your game in either direction.
This is the reason that you seem to do worse after good matches and better after bad ones. The particularly good or bad matches are rare, and because they’re rare it’s improbable that you’d have two in a row no matter what your coach does.
So don’t get discouraged when you can’t consistently come out on top multiple times in a row in any game of skill. You may be able to move your distribution up the right-hand side of the scale and/or squish it together so that there’s less variation, but you’re always going to regress to the mean somewhat because every round can’t be your best (or worst) round.
Now go give your dad a hug.
- Actually, in statistical parlance “average” is a vague term, but most normal people use it in the same sense that statisticians use the word “mean.” So let’s not make a big deal about it, okay? [↩]
- For the advanced students in the audience, these numbers refer to one, two, and three standard deviations above/below the mean [↩]
