Whenever I tell people about my work with statistics and the NCAA tournament, I get one of two reactions. Those who have little interest in college ball give me a patronizing smile and tell me some story about how their aunt won her pool simply by picking the cutest animal mascots. Then there are the clued-in hoops fans. They let me ramble on a bit before saying, “Oh, you're like a bracketologist.”
Most of the time, I'll just nod and say, “Yeah, sort of.” But if they don't wander away, I'll tell them the difference between bracketology and bracket science. Bracketology is the sort of thing that Jerry Palm, Joe Lunardi and Andy Glockner do. Leading up to Selection Sunday, they project who's going to make tournament, where they're going to be seeded and which region they'll play in. Their jobs involve prolonged analysis about who's punched a ticket, who's on the bubble and who's on the outside looking in.
Bracket Science is different. I don't care so much about all that. I'm more interested in examining which teams are going to advance in the tourney once the bracket gets set on Selection Sunday. My job involves intensive analysis about who's going to overachieve in the tourney, who's going to fall short of expectations and who's most likely to cut down the nets.
From a business point of view, my gig doesn't have the shelf life of bracketology. Speculation on who will be in the bracket goes on for months. Analysis of who will advance through the bracket has a big audience for about four days -- from Selection Sunday to the time when you have to submit pool sheet.
Eighty percent of my traffic on bracketscience.com comes in those four “bracket pondering” days. At some level, everyone wants to know the same thing: how can I build a winning bracket? Some people come right out and ask me what the answers are, as if I have them at the ready and am just holding out. Others are more appreciative of the predictive challenge before them and the limits of statistics. They like to think alongside me and use the numbers I offer to develop their own theories.
But my analysis gets pretty deep pretty quick, especially if you're giving yourself just a few hours to absorb all the guidance I provide and relate it to the 68 teams that make the dance. Last year, CBSSports.com did a great thing. They packaged a lot of my research into a handy tool they called the “Bracket Lab.” It gave people a quick read on the historical trends I've uncovered from the 28 years of data I have on the 64-team tourney era.
But many of the metrics I use still take some time to understand. And time is a precious commodity after Selection Sunday. CBSSports.com thought it might be a good idea to offer a few “Bracket Science 101” courses in advance of the dance. I agreed. Starting this week and continuing all the way up to tourney tip-off, I'll offer at least one class per week on the basics of bracket science. That way, when you conduct your experiments in this year's “Bracket Lab,” you'll be able to speed through the information much faster.
So let's get started building a better bracket.
What is baseline bracket prediction performance?
I'm here to tell you right now that nobody has a failsafe system for predicting the NCAA basketball tourney. As Nate Silver might say, filling out your bracket is an exercise in assessing probabilities. You're not going to build a perfect bracket. Not going to happen. There are nine quintillion different bracket combinations (two to the 63rd power or 9,223,372,036,854,775,808 to be exact). Sure, a lot of them -- like a Final Four of all 16 seeds -- are patently absurd, but even the number of plausible outcomes is astronomical.
Fortunately, your task is not to build a perfect bracket. It's just to win your bracket pool. Most people play in pools of 20 to 200 people. These pools are typically won by someone who gets between 48 and 54 correct picks out of the 63 games that start on the Thursday of the tournament. That works out to about a 75-85 percent accuracy rate. If that seems lower than you expected, remember that inaccurate picks in early rounds reduce the number of later-round games in which you even have a chance to be right. So three out of four actually ain't too bad.
If 48 out of 63 is the benchmark of winning bracket performance, what constitutes average tourney prediction success? When you consider that even the most clueless bracketeer could adopt a strategy of advancing the higher seed in all matchups, then that should stand as the baseline for tourney prognostication. In this case, every top seed would reach the Final Four. To determine winners in the semis and finals, it would be reasonable to advance the team with the higher victory margin, since statistics show that this is the best tourney performance indicator besides seeding. (We'll get into the value of individual attributes in an upcoming blog.)
I went back over the last 28 tourneys of the 64-team era and figured out how well the "higher seed/bigger margin" strategy would've done. Here's the year-by-year breakdown:
The numbers show that you should be able to predict about two-thirds of the 63 tourney games correctly without even thinking about it. In fact, the baseline bracket picking strategy would've correctly identified nine of the 28 champions in the modern tourney era, including Kentucky last year, North Carolina in 2009, Kansas in 2008, and Florida in 2007. That's four for the last six. Heck, if you had used the higher seed/bigger margin strategy in 2008, you would've correctly pegged the Final Four, called both the finalists -- and picked the winner.
Of course, this kind of late-round accuracy is an anomaly. Before 2008, all four top seeds had never reached the Final Four. And let's face it: picking your bracket solely by seeding isn't exactly a winning strategy. Using this approach would likely put you in contention for average-sized pools in only two of the 28 years -- in 2007 (49 correct answers) and in 1993 (48 correct answers). If you were playing in a smaller pool, maybe you'd be among the leaders in 2008 and 2009 as well.
Look no further than last year's dance for proof that seeding alone won't get you very far in your pool. Sure, picking Kentucky might've helped you land in the upper tier of your pool. But 41 correct picks out of 63 games is actually below average prediction performance. And 2010 and 2011 saw the worst two-year stretch of chalkiness since the 1989-90 tourneys.
There's no denying that seeding is the single best determinant of a team's tourney fate. It doesn't take a bracketmaster to figure out that the one, two, three and four seeds have accounted for 96 of the 112 Final Four teams and 26 of 28 champions in the 64-team era. The real the question is: which factors beyond seeding help improve your ability to predict tourney outcomes above the baseline accuracy of 65 percent? Before we can answer that question, we need a way to reliably measure the impact of attributes on tourney performance. That's where PASE comes in.
PASE: Performance Against Seed Expectations
If March Madness always played out to chalky seed expectations, the best one seed would win six games, the second best top seed would win five and the other two would win four. Then, the two seeds would win three games (before losing to top seeds in the Elite Eight), three and four seeds would win two, and five through eight seeds would win one.
Of course, the 28 tourneys of the modern era haven't played out that way at all. Instead of the average one seed winning 4.75 games, as would be the case in perfect high-seed dominance, they've actually won 3.38 games per tourney. Here are the average win totals of a single team at each seed position:
It's good to know these numbers as you make your bracket picks. If you think, for instance, that a second-seeded team is better than the average two seed, you might feel comfortable advancing them to the Elite Eight. But if you're convinced they're weaker than usual, it might make sense to have them bow out in the Sweet 16.
That's the most basic use for these “wins per team by seed” numbers. But they play a much more central role than that in my research. With these numbers, I've developed a metric to measure actual performance in the tourney to expected performance -- and assess the degree to which teams with certain attributes overachieve or underachieve.
This is the basic concept of “Performance Against Seed Expectations,” or PASE. Think of PASE as the equivalent of baseball's WAR, Wins Above Replacement. Since we know that the average “replacement one seed” wins 3.38 games per tourney, we can say that a top seed like Kentucky last year, which won six games, is 2.62 wins above expectations.
Using “wins per team by seed” values, you can tally the positive or negative differences between actual and expected wins for any attribute you want to study—whether it's an individual coach, teams with more than five straight bids, teams with all-Americans, guard-oriented squads, etc. The total of these differences is then divided by the number of appearances to arrive at an average number of games the attribute either beat or fell short of expectations per tournament.
An example will make this clear. Let's look at Butler coach Brad Stevens. He's gone to the tourney four times. In 2008, he won a single game as a seven seed. The average seven seed has won just 0.830 games per dance, so Stevens was .170 games ahead of expectations that year. In 2009, however, Stevens' ninth-seeded squad bowed out in the first round of the dance, falling .563 games short of expectations (that's the average number of wins for a nine seed). So in his first two trips to the dance, Stevens was actually an underperformer, falling .393 games short of the number of wins that teams with his two seed positions typically amassed, or -.181 games per dance. That was his PASE before the 2010 tourney.
Then Butler went on two magical runs to the finals. In 2010, the five wins that Stevens coaxed out of his Bulldogs were 3.839 games more than the typical number of wins a five seed gets in the dance. In 2011, Stevens was 4.304 games over projected eight seed wins Add the 2010-11 overages to Stevens' previous .393-game deficit and he's a whopping 7.75 games above seed expectations in four tourney trips, for a PASE of +1.938 -- nearly two games per dance better than his seed-projected win average. Overall, based on the seeding he had to work with, Stevens should've won just 3.25 games. He actually won 11. That's really, really good. To put it in perspective, Tom Izzo is the biggest overachiever among coaches with at least five tourney appearances, and his PASE is just +.761.
Coaching is just one attribute we can analyze in this way. Say you want to know whether Big Six schools that win their conference tourney actually overperform or underachieve in the national dance. You can get a precise measurement of that. Since 2002, when all the Power conferences had tourneys, the 66 champions have beaten seed expectations at a +.146 PASE rate. As you'll find out in succeeding blog posts, that's reasonably significant. Not Brad Stevens-like, but a solid PASE value.
So here's what we learned today:
• Baseline bracket performance is about 41 correct picks in 63 games.
• To win an average-sized pool, you need to get at least 48 picks right, an improvement in prediction accuracy of about 16 percent.
• Every seed has an average number of wins by team per tourney.
• With these numbers you can measure the degree to which any attribute contributes to over- or underachievement against seed expectations.
• The PASE stat is a key metric for assessing the relative values of attributes.
Next week, we'll start to analyze some of the basic attributes that people look at when they're filling out their bracket -- coaching, momentum, offense, playing efficiency and more. By Selection Sunday, you should have a good handle on the statistics that have historically correlated with the contenders and pretenders in the tourney.