The NCAA men's basketball committee released the long awaited replacement for the RPI, which it is calling the NCAA Evaluation Tool (NET).
The NET is described as a complex system that uses artificial intelligence algorithms to help create a ranking that more accurately predicts game outcomes. It is meant to be a combination of a results-oriented ranking system and a predictive one.
The NCAA says there will be five key components to the formula.
Team Value Index
Average offensive and defensive efficiency
Division I winning percentage
Adjusted winning percentage, giving rewards for road wins and penalties for home losses
Scoring margin, capped at 10 points. Overtime games will be treated as one point margins, regardless of the final score.
Team Value Index has yet to be defined, but it is results-oriented, involves strength of schedule and is the most dominant factor.
But that is hardly the only unknown part of the NET.
In determining teams' NET rankings, the devil is in the details and the biggest problem with this new system is that we may never get the details as to how these factors are used to create the rankings.
While the NCAA "will be as transparent as possible" with regard to defining and releasing the data that goes into the inputs of the system, according to NCAA senior vice president for basketball Dan Gavitt, no decision has been made as to whether the actual formula of how the NET is determined will be revealed to the public.
Gavitt was even reluctant to use the word "formula" to describe it. He said that, "formulas are archaic, like the RPI was" and said that the artificial intelligence algorithms may not be easy for people to understand.
So what if that is true? People should know specifically how they are being judged. Say what you want about the RPI, at least teams knew how to figure it out. If the algorithms (a fancy word for formula) are too complex for people to understand – and let's be clear, that includes the committee itself – then they are too complex, period.
The NCAA will make its rankings available, but releasing the formula is also important so that the results can be independently verified. Every year since the RPI formula became publicly available, I have found mistakes in the data that the NCAA used to calculate the RPI. I found those errors because I was able to duplicate the formula and investigate differences between my data and theirs. I was not the only one who found these errors. The NCAA relies on schools to self-report data that will be input into this system. Sometimes, there are mistakes, which are more likely to slip through the system if the results cannot be independently verified.
One hazard of using something that is expected to be much more accurate than the previous system is that people will expect the committee to rely on this data more. Why, in the name of accuracy, create something so complex that people may not understand it if you aren't going to follow it more closely?
Gavitt disagrees though. He says the NET will be used similarly to the way the RPI was used - as a means of aggregating teams into the buckets that are now being called quadrants. He said, "I don't believe the committee will necessarily rely on the NET more. It will be used as the tool to sort the data on the team sheet".
I think the committee will be in for a surprise this March though. The committee already gets a lot of grief when it doesn't follow Ken Pomeroy's rankings to the number. Now, those complaints will be louder when it does not put more weight on its own improved rankings.
The NCAA has worked very hard over the last decade to make the selection process as transparent as it can be without allowing us to watch the selection meeting. It would be disappointing now for them to create a new ranking system and go back to the days when the formula was a secret.