College basketball's coaches, its media, many of its fans, have long protested the NCAA to change its methods in how the NCAA Tournament is seeded and selected.
On Wednesday, the NCAA was officially swayed. The RPI is done. Finally! That flawed formula will no longer be part of how the best sporting event in America is assembled. The NCAA Tournament has changed forever; that's a really good thing. And it will now be built by committee members forming their opinions with a new metric called the NCAA Evaluation Tool. The NET.
Cute acronym, sort of complicated process. But ideally a much better, and fairer, way to constructing a 68-team field.
It's also just one metric. One metric -- that was built with the help of Google -- that was built after consulting how metrics such as KenPom, the Sagarin Ratings, ESPN's BPI and Strength of Record blueprints, and the emerging KPI were built. The NET has a lot of results-based data as part of its process, but there are also predictive elements.
Which makes the NET, at its core, a predictive-leaning model.
"It's a complete replacement," senior vice president of NCAA men's basketball Dan Gavitt told CBS Sports.
And, as betting markets and empirical data have shown over the years, predictive models tend to carry with them big-picture accuracy. In order for the NCAA to definitively declare that it was making an evolutionary step forward in how it ranks and sorts its teams for NCAA Tournament inclusion, going the predictive-model route was a must.
So the NCAA should be commended for making its leap, even if it's about a decade overdue. Even though, in what comes as a mild disappointment, the NCAA opted out of using a composite of multiple predictive metrics. That would have made for an even more accurate echelon -- but potentially complicated matters for the NCAA because it did not have ownership or control of each individual metric it would have been outsourcing for such a master ranking.
This new process was done in the spirit of making tournament selection and seeding better. Pure and simple. Let's get the teams that are most deserving into the field, and slotted properly. Combining this with the still-in-use quadrant system should allow for smaller schools to be more accurately represented on team sheets vs. what the RPI was holding back for so long. In theory, the formula could help mid-majors even if it wasn't built with any implicit bias in mind.
"There was no goal or intent, in any way with any segment of the game, to benefit or not, populations of teams," Gavitt said. "That goes both ways. We're trying to create something that is as fair as possible at every level of the the game. First and foremost, results of the games [matter most]. There was no intent, one way or the other. We're comfortable and confident that this is a fair and equitable metric for the entire game."
The NET, which is owned by the NCAA, will have five components that build out its 1-353 rankings. They are:
- Team value index. This is the most dominant feature, according to the NCAA, and it's based on win-loss results (with strength of schedule also being an important component). Opponent faced will be an emphasis here.
- Team efficiency. A team's average efficiency (points scored and allowed per 100 possessions) on offense and defense will be taken into account, which introduces predictive elements into the process.
- Wins. A team's overall D-I winning percentage will play a not-insignificant part.
- Adjusted winning percentage. This portion will reward teams for winning on the road, dock for losing at home, and balance out neutral-court performance. Opponents faced will not be factored here the way it is in the team value index.
- Scoring margin. For the first time, teams will be officially evaluated based on how much they win a game by. The scoring margin is capped at 10 points. This is a big step, backed by data.
While 10 points might seem like too low of an arbitrary cut-off, Gavitt said the NCAA and Google ran its NET model against thousands and thousands of game results, using scoring margin caps as high as 30 and as low as one.
"Having some scoring margin is important from a predictability standpoint, and it did make the rankings more accurate by having some level of scoring margin in it," Gavitt said. "Ten was the number that was the most optimal to getting the most level of accuracy without going so far that we started to influence behavior of coaches. The games' inaccuracy above 10 were very marginal."
Important to keep in mind regarding scoring margin: If games go to overtime, no matter how many overtimes and no matter what the final score is, the winning team will be assessed a +1 score and the losing team a -1 score as the margin in the eyes of the NET. The philosophy there being that teams played each other to a balance after 40 minutes, and overtime final scores can sometimes not reflect how close a game actually was.
Interestingly, according to Andy Katz of NCAA.com, "The committee did consider using the game date, an uncapped scoring margin, distance traveled and days of rest before a game but decided against using these in the equation."
In the eyes and minds of many statisticians, an uncapped scoring margin would have made for an even more accurate metric, but at least there's scoring margin to begin with. The fact this is the case can be attributed in part to college coaches as well, who were privy to these discussions and had constant feedback to the NCAA about how to kick the RPI out for good and bring in something that was much more sophisticated.
Mark Few, for instance, has been at the forefront of pushing for the demise of the RPI. He's done this despite being rewarded kindly, for the most part, with seeding over the past two decades as the head coach at Gonzaga. But on the whole, coaches I've spoken to over the past four-to-five years have been craving an overhaul.
"This metric, we are very confident it achieves what our intention was, and what the recommendation was from the NABC," Gavitt said. "It's a composite ranking from the standpoint that it's got a very significant foundation based of being results-oriented, because that is so important, we believe, to committee and coaches in the selection and seeding process. It has some levels of predictiveness to it now that the RPI never did."
It might be natural to see this news in mid-August and toss it to the side, but make no mistake: What the NCAA trotted out on Wednesday will be the talking point from February on of next year, when bracket forecasts become near-daily exercises and March Madness looms just over the horizon. By the time we get to March, this process will have its big moment on the big stage. Whatever the field of 68 looks like in March 2019, it should look undeniably different vs. what it would have been with the RPI still in place.
While the NET method is here to stay for good -- the RPI lasted for 37 years; with this new venture the hope is it can last just as long if not longer -- the NCAA is going to monitor its new metric very carefully, listen for feedback from analysts, coaches and others, and allow for tweaks to the NET in the years to come. If the 10-point scoring margin isn't as accurate as the NCAA believes now, and 2018-19's data provides evidence that 12- 14- or 15-point margins are better, an update could be made.
It's necessary to have that kind of flexibility.
As for the minutiae of what goes into the NET, and how each component is weighed, the NCAA has not yet officially decided on how or whether or not to release that information publicly. Whereas the RPI's archaic formula was always known, this one isn't yet. (It's obviously in the best interest of the NCAA to let the public see how the sausage gets made.) But the rankings will be public and updated daily, beginning in late November or early December. A Google employee will oversee the NET daily and keep tabs on its performance.
"We're going to be very transparent about the elements that go into the ranking and educate as much and as fully as transparently as we can," Gavitt said. "It's not really a 'formula,' so much as it's highly sophisticated and involves machine-learning is not easily digestible. It is not like the RPI, to consider this as a formula. This is not that. This is very contemporary, forward-thinking and involves machine-learning and artificial intelligence."
There's also a natural curiosity that bubbles up. OK, now we've got the NET. Can we apply it to previous years and see how its results compare to the RPI? The NCAA did that behind the scenes in building the NET, but it is not releasing that information for cross-referencing.
"We talked about using that lot," Gavitt said. "This is very much a contemporary, modern metric. The committee felt like there was nothing productive by going back and comparing it, frankly, to a system that no one liked and had any confidence in. It's a natural question. If I was a fan or a coach, I think I would be asking the same thing. Certainly we're going to be transparent about it."
Too bad. It would have been fun -- and given more credence to what the NCAA's helped build here.
At the end of the day -- well, season -- the basics of scheduling are still going to be most important. No metric can compensate entirely for poor scheduling in the non-conference. No system will rank a team so highly that it overcomes a miserable record. And, to my mind, it's best that we still have humans who get to take this information, interpret it as they do, and log their votes. (Though that process is also due for some upgrades, but we'll take this one tectonic plate shift at a time.)
"Keep scheduling tough," Gavitt said. "Have a quality strength of schedule. Whether it's the RPI or the NET or anything else, they're all fairly highly correlated. The committee is not going to change its philosophy in rewarding a strong strength of schedule. We'll be educating about that and engaging about that. This shouldn't be a sea change in terms of philosophy of scheduling."
Perfection is impossible. Improvement is not. Given how massive the NCAA Tournament has become, what the NCAA put forth on Wednesday amounts to one of its most important and consequential rule changes affecting men's college basketball in the past four decades.
Selection Sunday will never be the same, and college hoops is much better for it.
