NFL: Does the run really set up the pass?

The first round of the 2017 NFL Draft is in the books.

Imagine yourself in the shoes of an NFL team’s General Manager. Your objective for the upcoming season is clear: win as many games as possible. How? By assembling the best possible group of players you can put on the field. The draft is a crucial component of this process; it is a prime opportunity to bring players to your team who are, literally, game changers.

This draft in particular has presented GMs with an intriguing dilemma. Two star running backs, Leonard Fournette, and my alma mater’s Christian McCaffrey (clearly the better choice, in my 100% objective opinion) offer teams the chance to immediately upgrade their running games. Last night, each was selected in the first round – but will they prove worthy of their first round picks?

Prior to last night’s selections, GMs were likely struggling mightily with that very question. To help make such a decision, a GM might have turned to two common schools of thought: follow the example set by recently successful teams, or seek guidance from the long-standing truths of football lore. Or both, when the two happen to align.

Last season’s Atlanta Falcons, Dallas Cowboys, and New England Patriots come to mind as examples of recently successful teams, to say the least. Mixpanel Dashboards showcase how each offense was in the top 6 both in terms of points scored and yards gained in the regular season.

What did these offenses have in common? Excellent running games setting up high octane passing. Mixpanel’s Insights report elucidates just how often these teams ran the ball:

Atlanta ran, rather than passed, on 42% of its plays. That’s well above the league average of 39%. Dallas ran on a whopping 49% of its plays, the most in the league. Even New England, a team known mostly for its Tom Brady-led passing game, ran on 45% of its plays. And while these teams ran more often than most in the league, their passing games were nothing short of spectacular.

Atlanta quarterback Matt Ryan won the award for Most Valuable Player as he racked up close to 5,000 passing yards throwing aggressively into Devonta Freeman-softened secondaries. Dallas’ rookie quarterback Dak Prescott was remarkably efficient, showcasing some of the cleanest quarterback play from a rookie ever seen, thanks to Ezekiel Elliott drawing defenders. Tom Brady returned to 2007 form as he completed deep bombs set up by play action following steady runs from LeGarrette Blount.

In short, these recently successful teams were the embodiment of one of the most tried and true adages in football: “you have to run to set up the pass.”

As an NFL GM, you might be inclined to believe that these teams succeeded because they adhered to that “run to pass” philosophy. And having seen the results of Dallas bringing in Ezekiel Elliot in the first round of last year’s draft, you might very strongly consider Fournette or clear front-runner McCaffrey. Clearly, the Jacksonville Jaguars (my condolences, Leonard) and Carolina Panthers ascribed great value to these backs, quite possibly for those very reasons.

But does the data support that train of thought?

If I were in a GM’s shoes, I would want to be sure that a star running back’s impact would go beyond just the run game. I would want to be confident that he would boost my passing game as well – only then would I be comfortable spending my first round draft pick on him. And of course, I would want the data to support my decision.

Thus, I returned to Mixpanel to determine where the data stands on the issue. Applying Mixpanel to analyze Armchair Analysis’ data, we’ve already discussed why the Patriots were favorites to win Superbowl LI (I’m trying very very hard to not say “I told you so.” … Well, I tried.) We proved that, yes, Thursday Night Football is objectively awful. And we determined that icing the kicker really can impact kick success rate, depending on the distance. Now, it’s time to prove or disprove one of the most widely believed football proverbs.

To begin, we need to unpack the adage, “You have to run to set up the pass.” Traditionally, this refers to the tendency where if an offense executes a successful run for a significant chunk of yardage, the opposing defense will attempt to compensate by bringing additional defenders into the “run box.” The more bodies in the way of the run, the more likely it is for the run to be held short. However, if more defenders are in the box, that means there are fewer available to defend passes away from the box, so the passing game has greater opportunity to get the ball further down the field. If you need a visual, this NFL advertisement actually illustrates the concept fairly well.

Given that, we’ll need to look at passes that occur after runs. If we find that passes taking place after “good” runs are “better” than those that occur after “bad” runs, then we’ll have proven the adage.

But first, let’s define some terms. I’m considering any run that gains 5 or more yards a “good” run, since the average run in the past 16 years has gone for 4.19 yards. Conversely, a “bad” run is one that goes for 4 or fewer yards. Above-average is good; below average is bad. Similarly, we can evaluate the success of passes following these runs in terms of their yards gained—the more yards, the “better.”

With our hypothesis and terms defined, let’s dive into Mixpanel’s JQL Console, a tool that enables complex queries of Mixpanel data in simple Javascript.

My query is structured as follows:

Having seen the query, you might contend that the adage refers to the tendency of the run to boost a team’s passing game over the course of the 60-minute contest, not just in two-play sequences.

However, since we’re taking averages of these two-play sequences across entire games, teams, and seasons, we’ll still see the effects in the numbers. It’s true that not every good run will be followed by a high-yardage pass. But the average is influenced just as much by the high-yardage passes the adage predicts as well. So if indeed the adage is true, then the average passing yards after good runs should be significantly greater than the average passing yards following bad runs.

Without further ado, let’s get to the results:

As you can see, passes following good runs gain about 0.4 more yards than passes following bad runs, on average.

That might not seem like much, but consider the context – we’re taking an average of more than 700k runs and passes, where the average play, run or pass, gains 5.2 yards. So, 0.4 yards is about 8% of the length of an average play. Over the course of a game that typically sees at least 350 yards of offense, that 8% increase translates to an additional 38 yards. In a game where a score, and therefore the game itself, can be decided by mere inches, I’ll take 38 yards, or heck, 0.4 yards, any day.

Adage 1 – Skeptics 0

For the skeptics’ sake, let’s take the analysis a level deeper. Adding a couple lines to my query, I can also calculate the percentage of passes occurring after runs that are above-average in terms of passing yardage. An average pass gains 6.12 yards (including incompletions), so passes that gain 7 or more yards are “good.” 6 or fewer? Bad pass.

Interestingly, the percentage of passes that are “good” following either a “good” or “bad” run is just about equal, with only a very slight edge to passes following good runs. In other words, a quarterback is just about as likely to throw a good pass after a good run as he is to throw a good pass after a bad run.

Adage 1 – Skeptics 1

My suspicion is that the disproportionate difference in yardage stems from passes that either fall incomplete, or go for negative yards. Time to go deeper still! Updating the query, here are the percent of passes that are complete and go for positive yardage:

It seems the run has very little impact on the chance of a positive pass on the next play. Skeptics score 1. I ran the numbers on just completion percentage as well, just in case negative passes were skewing the results, but the difference was even less significant there: 61.2% completion rate following good runs, and 60.7% following bad runs. Skeptics score again.

Adage 1 – Skeptics 3

The evidence is certainly beginning to suggest that passing success is independent of the quality of prior running success. To put the final nail in the coffin, however, I wanted to confirm whether or not any play’s quality has an effect on the subsequent play’s quality. Therefore I built out the query to find all 16 possible combinations of the two play sequences: good/bad runs/passes -> good/bad runs/passes.

The results are enlightening:

A definitive blow to the adage. The numbers highlighted in yellow represent the average yards gained on a pass following a good run vs. following a good pass. A mere 0.02 yard difference indicates that the type of the previous play, be it run or pass, has almost no impact on a subsequent pass.

Instead, the success of the pass seems to be determined by the quality of the previous play, no matter its type. Note the figures highlighted in blue, representing the average yards gained on a pass following a bad run vs. following a bad pass.

In short, good plays tend to follow other good plays; bad plays tend to follow bad plays. The run sets up the pass no better than the pass sets up the pass.

Slipping on our GM shoes once again, we can feel better informed about the impact a star running back can bring to our team. Nothing we found in the data suggests that a strong running game translates to improvements in the passing game.

There is data to suggest, however, that the NFL is becoming a more and more passing heavy league. Mixpanel’s formulas report shows how many more passes are occurring than runs each year:

With everything pointing to the future of the NFL as a passing league, a GM needs to optimize his draft selections such that his team is ready to compete in that environment. So as tempted as I would have been to bring Fournette or clear front-runner McCaffrey to my team, I would have looked elsewhere for talent that will more directly impact my ability to compete for years to come.

That said, I can’t wait to see the duo of Cam and Christian shred opposing defenses. September 7th can’t come soon enough.

Artwork courtesy of Jack Kurzenknabe, and is in the public domain.