NCAA looking into changing the way the committee selects and seeds tournament teams

Changes could be coming to the NCAA's selection process

Some of the creators of the best known college basketball ratings systems, along with a few media folks with some knowledge and experience in this area, met with the NCAA on Friday to talk about how to come up with some better data to serve the NCAA Tournament selection and seeding processes.

This meeting came about as a result of a meeting the selection committee had with a committee of coaches representing the NABC this past summer. The coaches asked the committee to explore the idea of creating some sort of composite metric to replace the RPI, which the NCAA has used to help guide the selection process since 1981.

Whatever the committee eventually comes up with, it will not be in effect for this season. In fact, it's unlikely that any final decision would be made before the summer of 2017. If they can decide on something by then, it will be used for the 2017-18 season.

The RPI is primarily used as an aggregator. Teams are judged by such things as their records against the top 50, top 100, etc in the RPI. Strength of schedule is based on the RPI as well. The coaches would like to see a more sophisticated metric put into place.

The most discussed concept was how much the new metric should rely on margin of victory, if at all. The downside to including formulas that are heavily reliant on margin of victory, like Jeff Sagarin's and Ken Pomeroy's, are that teams do not play to maximize scoring margin, they simply play to win. That means that these ratings aren't exactly measuring what teams are trying to do. Also, the way that the end of basketball games is played can artificially increase or decrease scoring margin so that it no longer reflects the way the rest of the game went. Another downside is that if formulas that heavily rely on scoring margin are used, coaches will start playing to run up the score, even though as a long-term strategy, that probably won't impact those rankings very much.

Dan Gavitt, the NCAA's Senior Vice President of Basketball, emphasized that any new metric still needs to be results-based at some level. It is an explicit part of the selection committee's criteria that teams be judged based on results. That said, there is a lot of data to suggest that scoring margin systems do a better job of predicting winners going forward than results-oriented systems. Predicting winners is not necessarily the goal of the committee though.

Another thing that was discussed at some length was how to define a "quality win."

The committee has always broken down games by top 50, top 100, top 200 and 201+ in the RPI rankings, but they are open to the idea of redefining those lines. Pomeroy and Sagarin noted that such breakdowns would be redundant with their ratings because that is already a part of the rating. This discussion brought comments that the committee needs to do a better job of evaluating the quality of road win vs. a home win.

In the end, a lot of theories and ideas were exchanged, but this was a conversation that left more questions than answers. The biggest of those questions is what - exactly -- does the committee really want to measure? Until they come up with an answer to that, it will be impossible to proceed with any of the suggestions discussed Friday.