After I returned from the College Football Playoff mock committee exercise in Dallas last week, and after I wrote about my experience there, I felt like I hadn't really had a chance to think about the data the committee would be working with to make its decisions. When I participate in a mock committee for basketball, I already have an intimate knowledge of the data they use. I've been tracking it myself for over 20 years. But the football process is new –- to everyone. The committee itself has yet to use it for real. And the data that we were provided, which is exactly the same as the committee will use, left me feeling unsatisfied. 

There was no time to process the data there though. We were too busy using it to think about whether it was appropriate, although I did make a comment at one point that all those yardage stats are useless because yards don’t win games, points do. But use them we did, or some did, anyway.

So, I delved into the team sheets the football committee will use, and here's what I found ... most of the data is useless.

The mission of the committee is to "select the best teams" for the playoff and other bowls. The principles they state to help determine "best" are "conference championships won, strength of schedule, head-to-head competition, comparative outcomes of common opponents [without incenting margin of victory]" and other factors that could affect a team's performance, such as injuries.

Conference championships, head-to-head and common opponents are relatively easy to list, and when teams are compared, they are listed. Strength of schedule is not really addressed though.

Back when the committee came up with these principles, we were told that the committee would not rely on any one metric when during these meetings, like the basketball committee does with the RPI. I took that to mean they may look at a variety of metrics, but as it turns out, it means they are looking at none. There is no credible data provided to measure strength of schedule. 

Of course, you can see the schedule itself. The opponent, the opponent's record and the result of each game are listed. They also separate out wins against teams with records above .500 and losses to those below .500.  Other than that, all you get is the old BCS version of each team's collective opponents' record, and opponents' opponents' record. There are no other ratings.

That leaves strength of schedule to the eye of the beholder. It's wrong to leave something so important to the decision-making process undefined.

Instead, committee members get a litany of stats about each team that not only fail to help determine strength of schedule, they don't even help determine a team's ability to win games, and that's important too.

Instead, team sheets have data from 26 statistical categories, about 2/3 of which are yardage based. The others are points based, and there are a couple that combine points and yards but one can't use any of that to determine strength of schedule.

Only a handful of the stats provided correlate well to a team's winning percentage, and all of those are points-based. Naturally, there is some correlation to winning and the ability to gain yards or stop your opponent from doing so, but it's not nearly as strong as the scoring stats.

The best stat, by far, among those provided in terms of correlation to winning percentage is relative scoring offense. That means how many points you score in relation to what your opponent normally gives up otherwise. In 2008, Oklahoma was the top scoring offense at 54 points per game, but its relative scoring offense was 206 percent. That means the Sooners scored twice as much against their opponents than their opponents allowed on average.

However, an even more meaningful stat isn't even allowed in the room. Margin of victory.  Nothing correlates as well to winning percentage as MOV, but that isn't politically correct.

When we were discussing various data points in our mock meeting last week, a couple members brought up stats from other sources, and real committee chair Jeff Long stopped us from using them. He said since the formulas for stats that were brought up were unknown, they would not be allowed to use them as discussion points in the room. I actually admire that stance. People should know how they are being judged. However, using things that are misleading isn't any better.

So, while the basketball committee’s process has always been a subjective one, guided by objective, if not overly sophisticated data, the football committee's appears to be a subjective process guided by mostly useless and even somewhat deceptive data. And in one important area, there is no credible data at all. It's hard to see how that can end well.