GRAPEVINE, Texas -- I was a student journalist at Baylor back in 2014 when the final graphic on ESPN flipped to No. 4 Ohio State. The Bears, infamously co-champions of the Big 12 with TCU, found themselves sitting on the outside of the first College Football Playoff. Eight years later, to my great chagrin, the College Football Playoff mock selection committee did the exact same thing.
But we'll get to that.
Twelve media members, myself included, were invited to participate in the annual mock selection held by the College Football Playoff. The exercise takes place behind the doors of the same swanky meeting room at the Gaylord Texan as the actual committee that holds the fate of countless athletic departments in its hands.
For the day, I played the role of Navy athletic director and current CFP committee member, Chet Gladchuk. "Act a little crotchety to play Chet's role," joked one CFP staffer of the 72-year-old whose seat I inhabited. But if it was crotchety they wanted, a deep-throated rehashing of the controversial 2014 playoff was the perfect place to start.
The challenge of ranking the four "best" teams
The process for ranking teams has been extensively reported, but here's a quick rundown: The rankings are broken up into seven groupings -- every three spots 1-9 and every four 10-25. We vote to consider six teams at a time for each threesome. The first group: Alabama, Ohio State, Oregon, Florida State, Baylor and TCU. As it should be.
CFP chair and NC State athletic director Boo Corrigan charged us with finding the 25 "best" teams in the country. From the beginning, however, it became clear that each of us had different definitions of that metric. Some valued season-long success, while others wanted the best team as of Selection Sunday. Regardless, résumé ended up being a far more important consideration than which team would win on any given Saturday.
Debating criteria with some of the most famous names in the sports was perhaps the part for which I was most unprepared. The ex-players in the room were adamant that Florida State's undefeated season was a trump card. I disagreed. Try being a schlub like me telling Pro Bowl running back Deuce McAllister and former first-round quarterback E.J. Manuel that winning isn't everything because of the metrics.
The CFP has an analytic system created by SportSource Analytics that can compare up to four teams at a time with a wide range of metrics on giant screens in front of the group. When the Baylor vs. TCU vs. Ohio State vs. Florida State comparison came on the screen to delineate between spots No. 3 and No. 4, everything that happened in 2014 made sense.
Team schedules are displayed on the page with a list of results and common opponents or head-to-heads highlighted. However, the most striking part of the page is a color gradient with teams highlighted from green (good) to red (bad). When the schedules for TCU and Baylor came up, the amount of red vs. the rest of the field was glaring.
Never underestimate the power of data visualization!
Furthermore, one of the only true pieces of criteria listed by the committee is that similar teams with head-to-head victories or conference championships are given precedence. CFP director Bill Hancock confirmed to the group that since Baylor and TCU were both submitted as co-champions by the Big 12, the teams should effectively each be treated as having 0.5 titles.
When compared to outright champs Ohio State and Florida State, the decision wasn't difficult. We can argue about the specific strength of schedule and rating metrics that the committee uses, but it gave us a complete picture of the committee's decision.
Perhaps guided by Manuel's Florida State hand in the room, the 'Noles actually jumped up to No. 2 in our mock rankings, behind only Alabama. I voted Florida State No. 5; the 12 metrics that the CFP identified as most correlated with winning hated FSU, and its tight margins of victory weren't enough to convince me over other (better) conference champions.
Full disclosure, I tended to lean more heavily on the efficiency numbers and quality of play than other voters. SportSource Analytics neatly organized and color-coded stats that correlated most with winning after extensive historical research, with relative scoring and plays per point offense leading the way. (A special teams metric was added to the committee's top 12 stat factors at the behest of former Nebraska coach Tom Osborne.) Other mock committee members talked about the quality of their schedule. Some leaned on quarterback play, or tried to bring up NFL prospects.
Ultimately, that diversity of opinion is the point. Seven of the 13 CFP committee members are athletic directors. Seven are former college football players, including former NFL player and MIT Ph. D mathematician John Urschel. Two are coaches. One, former USA Today columnist Kelly Whiteside, is a journalist. Each one sees the game differently.
The combination of ex-players, writers and TV personalities gave our group similarly interesting context. San Diego State legend and NFL vet Kirk Morrison wanted to ensure the Group of Five contenders got requisite attention. The Athletic's Ari Wasserman asked us to consider recruiting gaps. AL.com editor John Talty implored us to remember that Katy Perry was in The Grove for Ole Miss' matchup with Alabama. McAllister and former Michigan quarterback Devin Gardner pushed us to consider winning above all. I brought up that Ole Miss was blanked 30-0 by a Bret Bielema Arkansas team that achieved a so-called "borderline erotic" bowl win over Texas. Everyone has their role.
How to rank the best of the rest
If you're wondering why there might be moments of inconsistency in a ranking, that's why. Every secret ballot features different voters weighing their own criteria while trying to convince other voters their perspective is right. And, really, that made the Nos. 7-25 conversation infinitely more interesting and competitive.
"When I was athletic director at Army, we would have been ecstatic to have made the top 25," Corrigan told the group. "We have to give the bottom of the rankings just as much consideration as the top."
That we did. The process demanded it.
Some attendees tried to bring conference strength into the picture, or dismiss some programs for being from the Group of Five. CFP executive director Bill Hancock and Corrigan quickly shot them down. After delving into the matchups and seeing the green-to-red scheduling gradient again, it frankly became clear that was not necessary.
The data visualization had records against teams above .500, against the committee's previous top 25 and previous top 10, but really, there were very few of those high-end data points from which to choose. Instead, schedules that were filled with green caught our attention. The ranked games -- or borderline ranked games -- were talked about on a case-by-case basis.
We pulled up Boise State and some were shocked by the amount of quality opponents in a "Group of Five" conference. It moved up. Conversely, Wisconsin's "Power Five" schedule was filled with red. SEC contenders Ole Miss and Georgia were filled with green wins and red losses. Michigan State and Kansas State had no green wins, but their only blemishes came from the nation's top teams.
After each round, we'd vote in new teams. At the end, we had an opportunity to discuss any great disparities. UCLA originally ranked behind an Arizona State team the Bruins walloped in Tempe, Arizona. That was remedied. Georgia moved further down, while Arizona rose multiple spots. Marshall got into the field after comparing well to teams like Minnesota and Louisville in the 12 factors.
This discussion took place over five hours, by the way. The real thing features six weeks of talking, many hours of debate and countless weekends of watching cutups of every relevant football game imaginable. Hancock noted that there was one committee member that decided to individually grade each player in every game that he watched. Every year is different.
What will change (and what won't)
Naturally, that leads us to the 12-team playoff. In many ways, the decisions that the committee makes are about to change. With six auto bids, there will be less emphasis on figuring out which teams make the field and more emphasis on ranking and seeding teams.
When the playoff expands, the committee will remain at 13 members. They still plan to rank 25 teams and feel comfortable that at least six conference champions will fall within that number. Instead of filling bowl pairings with teams down the bracket, those will now fill at-large playoff spots. Basically, the process will not change much whenever the playoff expands, which those in the room hope is sooner rather than later.
Ultimately, I gained a lot of respect for the process. It's not perfect, it's not unimpeachable, but it's thorough. With the amount of information and video available to the committee members, I feel more confident that they have all the tools necessary to make the best decisions possible. More importantly, the secret ballot process makes jimmying the rankings for branding or matchup purposes just a little more difficult. Transparency won't assuage any conspiracy theories, but it makes me more confident in the process.
And to Hancock, whenever you need a new name to add to your list of committee members ... I'm available.