Mixpanel’s game theory: Does icing the kicker work?

“The difference between ordinary and extraordinary is that little extra.”

—Jimmy Johnson

It’s a different sort of “Hail Mary”, but it’s what coaches and teams hope for: that slim chance that the kicker’s rhythm and mental preparation will be disrupted by a last-minute timeout, a tactic known as “icing the kicker”.

The impressive amount of research and number-crunching on the topic of “icing the kicker” has done little to answer whether it’s actually an effective tactic or not. Both the results of this research and opinions vary widely:

• Grantland and Freakonomics say icing doesn’t work
• The American Statistical Association says icing probably works, especially for longer kicks
• ESPN says icing has the opposite effect; it actually increases the chances of a successful field goal

And yet, ever since a 2004 rule change that allows NFL coaches to call timeouts from the sidelines, the practice of “icing” persists. Whether for reasons rooted in game theory or superstition, icing isn’t going anywhere – even if it doesn’t make a difference. Heck, even Madden 17 added an ice-the-kicker feature this year.

We know that icing the kicker has worked at least once this season. And deciding not to ice the kicker has failed at least once. But how often does this traditional tactic sway an NFL game? Is the metaphorical coin flip a chance worth taking?

We’ve already used Segmentation and JQL on fifteen years’ worth of in-game NFL data to understand why the Patriots were favored to win the Super Bowl in the preseason and why Sunday night games are so much better than Thursday night games. Could we use Mixpanel again to settle, once and for all, if icing the kicker ever makes a difference in a game’s outcome?

To ice or not to ice; let’s look at the data

To answer this question, star Support Engineer Brandon Skerda again took matters into his own code. He built a JQL query to see how successful icing the kicker has been in the 21st century.

We’re going to get technical for a minute and lay out how we built this query. If you want to skip to the results, just scroll on down.

To begin, we considered when a head coach might want to ice an opposing kicker. Timeouts in the NFL are incredibly valuable and as such, given the anecdotally minor chance of the timeout impacting the kicker’s performance, a coach would likely only consider an ice in desperate situations, with the game on the line. These situations can be defined by the time on the clock and current score of the game:

• The game must either be in the last minute of the fourth quarter or in overtime, where a field goal is potentially a game-winning event

• The pre-kick game score must see the offense anywhere between three points behind, and six points ahead of the defense
  • If the offense is three points behind, a successful field goal would likely send the game to overtime
  • If the offense is six points ahead, a successful field goal would likely seal the game by forcing the defense to score twice (at least nine points) in very short order

Next, we defined an “ice” in the context of our data. As discussed in our earlier NFL data explorations, our dataset consists of one datapoint per play, with each datapoint containing numerous properties of the play in question. One of these properties is “Timeouts Remaining for the Defense.” If a situation arose with the game on the line, where the defense took a timeout between the previous play and a field goal, we considered that timeout taken to be an ice.

Definitions in place, we constructed the query. “First, having queried all 700,000 plays in our dataset, representing every play from the 2000-2015 NFL seasons, I then filtered down the data to just those plays where the game is on the line,” Brandon says. The first filter: return events from the fourth quarter or overtime.

Next, for fourth quarter plays, only consider those in the last 80 seconds (enough for two full-40-second-play-clock plays, one of which being the field goal attempt). Then, we filtered again to address closeness of score, removing all plays where the outcome of the field goal would not decide the game.  

Now we can dive into the real meat of the query. To start, we iterated over all the plays that passed through our filters, seeking out sequences of two plays in duration where the second play is a field goal attempt. We tackled this task on a per team, per game, and per quarter basis (as either team might have opportunities for a game deciding field goal in both the end of the fourth quarter and overtime).  

Having identified these sequences, we saved the relevant details of the plays: their name (“Pass”, “Rush”, “Field Goal”, etc.), the time on the game clock, and the number of timeouts remaining for the defense.  

Taking these details into account, we can check for the final condition that determines whether or not the field goal attempt was iced: if the defense had more timeouts remaining on the play prior to the kick, the kicker has indeed been iced. With that, we can determine whether or not each sequence contained an ice.

Next, we grouped up each sequence into four possible categories:

1. Iced and Good – the kicker was iced, but made the field goal, earning three points for his team
2. Iced and No Good – the kicker was iced, and much to the delight of the defense, missed the ensuing kick
3. Clean and Good – the kicker was not iced, and, perhaps unsurprisingly, made his kick successfully
4. Clean and No Good – despite not being iced, the kicker missed his field goal attempt, again much to the delight of the defense

Finally, it’s just a matter of math, counting the number of kicks in each category and performing some simple division to determine the average success rate of kicking a field goal in iced and clean situations. The results are...anticlimactic. Having crunched all that data, it seems kickers that have been iced are a whole 0.1% less likely to make their kick successfully. While these results suggest that icing the kicker may not be effective in game on the line scenarios, there must be some reason why head coaches turn to the tactic.  

Thus, we broadened our search to include all field goal attempts, at any point during a game. After all, the game need not be on the line in order to necessitate an ice. Consider a last second field goal at the end of the first half: if the team on defense has a timeout to spare, with little possibility of a successful drive after the kick, why not burn that timeout in the hopes of saving three points? Having removed the game-on-the-line filters, the results began to shake out a bit more.

Across all field goal attempts, the gap between success rates widens to 5.4%. Of course, though, we’re comparing very different sample sizes for clean and iced kicks: 12,325 total field goal attempts to 541, respectively. Given the relatively small number of ices, these results must be taken with a grain of salt. However, since icing the kicker only became possible slightly more than a decade ago, we’re looking at all possible occurrences of icing the kicker.  

If I’m a head coach, 5% is not terribly compelling, especially given the questionable statistical significance of the data. After all, my timeouts are precious, as they give me the ability to march my team down the field without taking too much time off the clock. It’s therefore a careful balance: do I want to give myself a small chance of sealing the game right now, or preserve more time for my team to work with on an ensuing drive?

Let’s think about this a little more. If the opponent scores a field goal, the ensuing kickoff will more than likely leave me behind my own 30 yard line. At that distance to the goal line, my chances of scoring are low. According to to Brian Burke of Advanced Football Analytics, I have a less than 20% chance of getting a touchdown, and a less than 10% of getting a field goal. That likelihood is further reduced in time-pressure situations like the end of a half, but I would still need more than 5% from an ice.

Field goal success rate, of course, is heavily dependent on the distance of the kick. Until now, we’ve left that variable out of the equation, but it most certainly could have an impact on the results. Therefore, we grouped “clean” kick vs. “iced” kick success rates by the distance of the field goal attempt – a simple change to the JQL query.

These trend lines tell an intriguing tale: icing the kicker starts being effective (meaning, the opposing kicker becomes more likely to miss their attempt) at 33.9 yards, and becomes increasingly more effective the larger the distance between the ball and the goal posts. At long ranges especially, the trend suggests that icing the kicker considerably increases the likelihood of that kicker missing the field goal attempt.

To sum it all up…

First off, let’s look at the results that we expected: In non-critical situations, where the game isn’t on the line, kickers are successful 82% of the time and do worse when they’re iced (82% overall success rate for kicks vs. 77% success rate for iced kicks).

In game-critical situations, kickers have a 76% success rate; meaning, the increased pressure makes them worse overall. But, as Grantland concluded five years ago, icing actually seems to have no effect on a kicker’s success rate, overall.

But, if the distance of the field goal is 34 yards or greater, icing at any point in the game – including critical game-on-the-line situations – does, in fact, make it less likely the kicker will succeed. Not surprisingly, the greater the distance, the higher the odds of a kicker missing.

Therefore, depending on the situation, there are a few conclusions a head coach should make based on this data:

1. If there is enough time to make a reasonable attempt at a drive, the decision comes down to some slightly more complicated math. With the results above, we can determine how likely an ice is to cause the kicker to miss their attempt. The probability of scoring on the ensuing drive is highly team-dependent. But if the chance to save the field goal is larger than the chance to score, a head coach should ice away.
2. If there’s not enough time to make a reasonable attempt at a drive following the kick-potentially-to-be-iced, and at least one timeout remains, the decision comes down to the distance of the kick entirely. Any more than 34 yards and the ice is, statistically speaking, likely to reduce the opposing kicker’s chance of making that field goal.

The Vikings faced the latter situation against the Lions in Week 9. Up three, a Lions field goal sends the game to overtime. There would be no time for a last minute Vikings drive – only five seconds remained on the clock. The Vikings had one timeout left as Matt Prater readied for a 58-yard attempt.

“Brandon Skerda, head coach of the Minnesota Vikings, ices the bejeezus out of Matt Prater in that situation,” says Brandon. “A 58-yard field goal has a 24.5% chance of succeeding if not iced and 10.0% chance if it is. The Vikings would have been 59.5% more likely to win the game, avoiding overtime altogether, if Mike Zimmer had iced the kick.”

If football is a game of inches, that 59.5% increase in the chance of winning sounds like a mile. 

The article was co-authored by Brandon Skerda, who really is a rock star Support Engineer at Mixpanel and our resident NFL & NBA analytics expert. Connect with Brandon on Twitter: @bskerda