Posts Tagged ‘statistics’
Regression to the Mean and Owning Some Chumps
Let’s say that you’ve hired a coach to help you improve your Slayer game in Halo 3. I’ve heard of stranger things. Let’s say this coach looks like Mr. Miyagi but he curses WAY more. He uses a variety of training and motivational techniques, ranging from grenade throwing drills to trigger finger sprints, doing everything he can to drive you towards perfection. You notice, though, that he eventually stops praising you whenever you rank at the top of a match. He did at first, but now when you earn more than your usual number of kills your coach stands stoically by, straight faced and not giving you a single word of praise for those outstanding rounds.
Eventually, you ask him why he never praises you when you do a really good job.
“Because,” he says, “I’ve noticed that praise doesn’t work. Every time I praised you for a really good round, your next round is always mediocre. And what’s more, when I yell at you for playing poorly, next round you always do better. Praising not only doesn’t make you better, it makes you worse.”
You pause for a second, then cry “You’re not my real dad!” and run out of the room, bawling like a child. Yes, you do. That emotional outburst aside, though, is your coach’s logic sound, given that you DO in fact perform worse every time he praises you for doing well and perform better whenever he rebukes you for doing poorly? Praise makes you do worse and berating makes you better, right?
Nope. Your performance following stellar rounds of Halo or Starcraft II any other game involving skill can be best explained not by the effects of praise or punishment, but by something called “regression to the mean.”
Let’s assume that if we looked at your performance over a bunch of matches and plotted them out with ending scores along the X axis and how often you end a match with that particular score on the Y axis. They’d probably form something close to a normal, bell-shaped distribution like this:

Figure 1: Your sick skilz, plotted
If we were to pick any single match at random, it’s more likely that your performance would be about in the middle somewhere –somewhere near the “mean,” which is basically another word for “average.”1 In this example, that’s 10 frags. It’s rare that you’re at the very top (17 frags) or bottom (3 frags). In fact, if your performance follows a normal distribution like the one above, then the following will be true:
- 68% of your matches will end with scores between 8 and 12
- 95% will be between 6 and 14
- Only 0.6% will be under 4 or over 162
And even if your distribution is a little skewed because you do well more often than you do poorly, the numbers won’t change much until things get REALLY skewed. At which point no amount of coaching is going to change your game in either direction.
This is the reason that you seem to do worse after good matches and better after bad ones. The particularly good or bad matches are rare, and because they’re rare it’s improbable that you’d have two in a row no matter what your coach does.
So don’t get discouraged when you can’t consistently come out on top multiple times in a row in any game of skill. You may be able to move your distribution up the right-hand side of the scale and/or squish it together so that there’s less variation, but you’re always going to regress to the mean somewhat because every round can’t be your best (or worst) round.
Now go give your dad a hug.
- Actually, in statistical parlance “average” is a vague term, but most normal people use it in the same sense that statisticians use the word “mean.” So let’s not make a big deal about it, okay? [↩]
- For the advanced students in the audience, these numbers refer to one, two, and three standard deviations above/below the mean [↩]
