The week before last was projections week on the Fantasy Football Today podcast, and Heath Cummings and I broke down how we see the 2020 NFL season going for every team. Projections have tons of utility, and simply going through the process of questioning your assumptions for each team can lead to a ton of actionable information for Draft Prep season.
At the same time, I've long believed projections are too heavily leaned on in Fantasy football drafts. You'll frequently hear analysis centered around the most likely expected outcome for a given player, and the general discussion of draft results and things like automated draft grades will typically rely on these expected outcomes.
The weight projections are given both makes sense — they will always reflect our best guesses for the coming season after an offseason of personnel moves — and is counterintuitive. It's counterintuitive because Fantasy leagues are won and lost on the biggest performances each season. Lamar Jackson's upside was noticeable in a 2019 projection, but no projection compared to the actual league-winning performance we observed on the field. He's one of a hundred examples — injuries changed the circumstances for some teams and players, others just got better or worse.
Where projections are a bit more reliable in some other sports, a key element that adds to the difficulty in the NFL is the evolution of the league as a whole. It's a copycat league, and scheme and intent play more of a role in the stats we see than in other sports. Major shifts like increased three-point shooting in basketball or the emphasis on launch angle and swinging for power in baseball are commonplace on an almost annual basis in football — whether it's more passing, an emphasis on certain personnel groupings, more use of the shotgun; sometimes it's a cluster of teams trying something, sometimes it's a league-wide shift.
The only thing reliable about the NFL is it will be unreliable. That figures to be truer than ever in 2020, amid the Coronavirus pandemic, which has already impacted offseason training programs, will impact training camp, and may even cause the entire preseason slate to be canceled.
And this is where the over-reliance on projections comes in — one major complaint about them is despite the dozens or perhaps hundreds of projections out there, they don't tend to differ all that much. That's in large part because the best we can do is rely on the data and information we have, and most people going through projections are taking a fairly similar approach with that data.
I'm not advocating to ignore projections entirely. But actively understanding the ways every NFL season differs from a set of projections is a key edge in drafting; I'd argue it's one of the biggest edges left. If the issue with projections could be summed up in a simple concept, it's that they are too dependent on past observed outcomes. Their utility in identifying changing trends is clear, and yet the results of a projection are still limited by those prior results.
And there's no easy answer here, as I will demonstrate below. It could be said that those doing the projecting should be more willing to take a stand and visualize major shifts, but the simple fact is the further you get away from prior trends, the more likely you are to be wrong. Staying reasonably anchored to observed data tends to make projections more accurate in the aggregate, but it also makes league-winning outliers impossible to project.
Let's dig into my 2019 projections and check out some of the biggest misses and what they meant. For simplicity, I'll largely stick to team-level assumptions and projections. Everything on the player level flows from the play volume and run/pass split of an offense, but rest assured there were many more errors made at the player level, divvying up targets and rush attempts.
My projection was within 10 total plays of the actual 2019 play volume for eight of 32 teams, which seems solid. But I was also at least 50 plays away from the observed total for nine teams. In terms of run/pass, I was off by at least 50 pass plays for 11 teams, and at least 50 rush plays off for seven.
For exactly half the teams, my projection was at least 50 plays, passes or runs away from the actual observed outcome. That's a ton of missed opportunity, and it reinforces the clear takeaway — I shouldn't be too sure of any of the assumptions I made in my 2020 projections.
Here are nine examples of teams I missed on where past results and other context made it difficult to foresee the end result.
- Ben Roethlisberger played just two games, and I was 111 plays and 111 pass attempts higher on Pittsburgh's projection than their actual results.
- Cam Newton also played two games, and I was 57 plays and 82 pass attempts lower than Carolina's results.
- Dallas became pass heavy. They added 47 plays and 70 passes to their high marks from 2016-2018.
- I probably didn't adjust my Colts projection enough after Andrew Luck's retirement, and missed on their run/pass ratio by five percentage points.
- Melvin Gordon's holdout helped push the Chargers from a team that was within two percentage points of a league average rush rate in each of 2016, 2017 and 2018 to one that was nearly five percentage points more pass-heavy.
- The Raiders clearly wanted to run after drafting Josh Jacobs in the first round, but betting markets didn't see them as a very good team, which would have made that difficult. A 6-4 start helped them fulfill that goal, and they were substantially more run-heavy than I expected.
- San Francisco was projected to be improved with an win over/under of 8, but their 13-3 season kept them in obvious run situations. Their rush percentage was much more extreme than the slight run lean I projected.
- Jacksonville was a 5-win team in 2018 that still maintained a slight run lean, but they abandoned any pretense of the run-and-defense philosophy that helped them to the 2017 AFC Championship and became a pass-happy 6-win team in 2019.
- Washington ran fewer than 900 plays after at least 967 in each of the previous three seasons, notably going extremely run-heavy late after Jay Gruden was fired.
Teams like these make it clear that we can't trust past trends to be predictive. Quirky things happen in NFL seasons, and they can shift the projected team atmosphere considerably.
Outside of these teams, though, there are seven more I missed on that I want to dig into. In some cases I will be referencing notes I took on the assumptions I made at the time I was projecting the offense, and as you'll see, my errors were not all similar — there were examples where I went out on a limb but was wrong, there were examples where I identified a likely shift but wasn't aggressive enough, and there were examples where I missed something completely. But perhaps most commonly, my errors boiled down to ascribing too much certainty to something uncertain, which follows the above examples and is an inherent flaw in projections because decisions have to be made for a final result to be reached.
Projected plays/passes/runs: 1007/562/409
Actual plays/passes/runs: 1096/684/362
In my notes, I wrote that the Falcons' recent trends were "very hard to parse." The league average rush rate is typically around 41%, meaning of all plays, 41% are runs. Atlanta was over 42% runs in 2016 and 2017, then very low at around 35% in 2018. I noted there were some personnel changes — the emergence of Calvin Ridley and departure of Tevin Coleman — but that significant injuries to Atlanta's defense in 2018 had a big impact on their pass lean.
In the end, I projected them to regress their rush percentage closer to league average, but to still give them a lean toward more passing than average. So I basically split the difference between 2018 and their prior trends. I may not have considered the offensive coordinator change strongly enough, and the Falcons wound up even further away from the league average rush rate at 33%, the biggest run lean in the league.
Their play volume also exploded — after three seasons between 984 and 1,010 plays, I had them at 1,007. They ran 1,096 plays, second most in the league, and I was off by more than 100 pass attempts.
Projected plays/passes/runs: 965/530/388
Actual plays/passes/runs: 1022/615/349
Miami's results were similar to Atlanta's in some ways, but very different in others. In 2018, Miami became the first team since 2006 to run fewer than 900 plays. With a new coaching staff on hand, I added almost 100 plays to their 2019 projection, but was still 57 plays shy of their actual 2019 result. I noted Miami's team-level data was "tough with a new staff" but a "negative script should mean a pass lean" despite them inexplicably being more run-heavy than league average in 2018 under Adam Gase.
In the end, Miami's 34.1% rush rate was one of the biggest pass leans in the league, and their team-level aggressiveness opened the door for DeVante Parker's big season.
Lessons learned: It's difficult to project an extreme change in volume or run/pass split, but I did note these teams were hard to read. Ultimately, I should widen the potential range of outcomes for everyone when it's clear there's no obvious trend. In 2019, Washington became the second team since 2006 to run fewer than 900 plays, and with a whole new staff that just ran a faster-paced team in Carolina in 2019, I'm open to Washington's play volume and run/pass ratio changing drastically in 2020.
Expected too much
Projected plays/passes/runs: 1046/585/423
Actual plays/passes/runs: 973/539/393
Good teams that sustain drives and score a lot of points tend to run more plays, and I made it a point to project positive things for an incredibly talented Browns offense last year. I noted I chose to go up tempo with a slight pass lean due "because of (Todd) Monken's influence." Interestingly enough, my 40.4% rush percentage was right on the money as Cleveland hit that 40.4% mark exactly.
But where I decided to jump Cleveland to what would have been four-year highs in play volume, both passes and runs, it was widely reported that Monken didn't have much influence at all as offensive coordinator, and the Browns stumbled under head coach Freddie Kitchens. They wound up with four-year lows in plays and pass attempts.
Projected plays/passes/runs: 1034/585/403
Actual plays/passes/runs: 1000/554/396
Arizona was one of the teams I got within 50 plays, passes and runs, of. My projection wasn't way off. But it's interesting because in 2018 they finished with just 902 total plays, and I projected a massive increase under Kliff Kingsbury. Unlike with Miami, where I undershot, I overshot this one, and I wound up projecting an offense that was faster-paced and threw more passes than league average. They wound up below league average in both regards.
Lessons learned: Going out on a limb with an overly optimistic projection for an offense can be way wrong in the case of Cleveland, or there's such a thing as partial credit in the case of Arizona. Indianapolis and both Cleveland and Arizona again are all examples of teams I'm projected for better things in 2020 than they've shown us recently, but I shouldn't trust optimistic projections too much because it's possible to swing and miss.
Didn't expect enough
Projected plays/passes/runs: 1045/562/451
Actual plays/passes/runs: 1055/632/401
I wrote in my notes I was "expecting a bit more passing than the rush lean they've shown because of (Todd) Gurley's health." The Rams in 2017 and 2018 under Sean McVay had leaned about two percentage points more toward the run than league average, some of which was due to their success, as winning teams run more. I still projected them with a run lean, but closer to league average.
As it turned out, they leaned more than three percentage points toward the pass, and while I was close on their up tempo nature leading to more play volume than league average, I was substantially off on their pass and run totals. Cooper Kupp's big first half and Tyler Higbee's big finish are ways that increased pass volume manifested in Fantasy value.
Projected plays/passes/runs: 1026/464/519
Actual plays/passes/runs: 1064/440/596
Everyone knew Baltimore would be more run heavy with Lamar Jackson at quarterback and Greg Roman taking over as offensive coordinator. I noted "Greg Roman offenses have without exception been run heavy and below average in pace, mostly quarterbacked by good Lamar proxies (Colin) Kaepernick and Tyrod Taylor."
I still projected Baltimore to run more plays than league average. The 50.6% rush percentage I projected was nearly 10 percentage points higher than league average, and would have made them the second-most run-heavy team since 2016. Instead, they were the most run-heavy team by more than three percentage points, finishing with a rush percentage of 56%, and their offensive success helped them run substantially more plays than league average. They also became one of just four teams since 2014 to throw fewer than 450 passes.
As hard as I may have tried, there was just no way to project something as extreme as their 2019 offense. Some of that was due to not recognizing just how dang good they would be.
Projected plays/passes/runs: 1015/548/430
Actual plays/passes/runs: 970/466/476
My projection added nearly 80 rushes for 2019 to Minnesota's 2018 total, and I was very much on board with the Vikings turning into a heavier run team. But I still wasn't even close, as they became one of three teams (along with Baltimore and San Francisco) to run more times than they threw. They lost 140 pass attempts from their 2018 ledger, and were 60 passes shy of their far run-heavier 2017 season.
The heavy run lean helped crater their play volume as well, and I was significantly too high there after they had hit at least 1,000 plays in three straight seasons (league average in 2019 was 1,016).
Lessons learned: Whereas I tried to read the tea leaves with the Browns hiring Todd Monken and the Cardinals hiring Kliff Kingsbury but overshot their offensive changes, the changes I made for these three teams — projecting major shifts from their recent trends — were not aggressive enough. The lesson is that even if you recognize shifting trends, the degree of the change is still very opaque.
Even teams like the Saints and Patriots, with longer track records, saw significant changes last season. For both, I projected a solid run lean, but both wound up slightly more pass heavy than league average in 2019, which was out of line with recent trends. Of course, there were wins, too — I almost perfectly nailed the Packers' shift toward more running, and my four-year high pass total and four-year low rush total projections for the Bills were both within 10 of the actual results.
But it simply couldn't be more clear that projections are an inexact science. There are better projectors than me out there, but even the best wound up closer to my numbers than the actual results for many of the above situations. That's because while past is prologue, and it needs to be respected when projecting, the actual outcome of any NFL season contains many significant shifts that are much easier to recognize in hindsight than they are before the season starts.
And yet, tons of Fantasy draft tools and draft analyses are built around projections. As I said at top, an overreliance on projections is one of the biggest edges left in Fantasy. The trends we've seen over the past few seasons will shift in 2020, as they always do. New stars will emerge, and old ones will fade out. We'll have more in the coming weeks about how you can take advantage of these realities, but for the time being, keep yourself open to unlikely possibilities when drafting, and please don't sweat that bad automated draft grade.