Data Insights

Mixpanel Decodes the NFL

Jennifer Carney Writer of things, interviewer of literal and figurative rock stars.

“It’s not whether you get knocked down,
it’s whether you get up.” 

Vince Lombardi

The 2016 NFL season is off to a rocky start. Ratings are down double digits across the board. A retired future Hall-of-Famer (and major viewer draw) is now a part-time pizza spokesman. Nobody’s sure whether to panic or to R-E-L-A-X.

When I asked coworkers how they felt about the NFL season so far, the reactions were mixed. Some were pumped about early season thrillers, others felt bored by the games. But there was one opinion that united us: the belief that football on Thursday was the most boring and football on Sunday was the most exciting.

That got me thinking: Could we use Mixpanel to prove that Thursday night games were objectively worse than other NFL games, and that Sunday night games were objectively better?

This seemed easy to prove on its face. Thursday Night Football has always struggled against Sunday Night Football for a ratings share. Sunday Night Football has held the top-rated program spot three times in the last five years.

Even when SNF is up against one of the most-watched presidential debates in U.S. history, it still trumps TNF: Week 5 Sunday Night Football had a 10.2 Nielsen rating, but Week 5 Thursday Night Football came in at just 6.7, although it edged up to 9.9 for Week 6’s Chargers/Broncos contest. (For context, the second presidential debate had a 10.4 rating.)

But ratings are just one facet of the NFL’s success. We had a hunch that there was more to the story hiding in actual in-game data.

To take an analytical approach to our gut feeling, we abandoned Nielsen ratings and focused on gameplay instead. From there, we decided to define and compare certain metrics for games on each day of the football week. If Thursday Night Football performed worst out of all these games and Sunday Night Football performed best, we’d uncover the “why” behind the ratings dip and put some data behind our hunch.

tnf-3c-ctt

Pre-Gaming

Before we did a data deep dive, we did an exploratory analysis. We suspected that TNF fields such bad match-ups that not even Color Rush could persuade people to watch in greater numbers than they watch “Football Night in America.”

The way we (retroactively) defined “good” and “bad” teams was simple: greater than 8 wins = good team, 8 or fewer wins = bad team.

We won’t fill this post with digital reams of data; last year’s TNF and SNF match-ups are representative of the overall trend:

tnf-1

snf-1

Seeing that Thursday Night Football more heavily skewed toward featuring average/bad teams — and Sunday Night Football featured many more games (including the SNF-branded Thanksgiving night game and the NFL season opener) between two good teams, fewer mismatches, and an equal number of bad vs. bad — we decided to press on with our analysis.

Defining “good” and bad

To answer the question, are Thursday Night Football games objectively worse than all other NFL games? we first had to define the most subjective of parameters: good and bad.

In the same way fans recognize a great game for being relatively better than other games, a sucky game is relatively worse. But this is Mixpanel, and that’s not a very data-driven approach.

I enlisted the services of star Mixpanel Solutions Engineer, Brandon Skerda, once again. He went back to the initial dataset we used for our Brady-less Patriots analysis, which contains the details of every single play, from every single game since Thursday Night Football came into being (2006) to see how we could define “good” and “bad”.

Now the question became, what makes one game any better than any other game?

All NFL fans want to see competitive games. We want to see big plays on both offense and defense: multiple touchdowns, turnovers, and sacks. We want to be fired up for the entire game.

So what, then, makes a game objectively better or worse, and how would we be able to describe that when our data set is composed of individual plays within these games?

Since there is no universal definition of “good” or “bad”, we devised our own definition of “good” and “bad” games. Here’s how it breaks out.

A “good” game is one that satisfies all of the following criteria:

It’s competitive. At the two-minute warning, it should be a one-score game, so the leading team can be up no more than eight points on its opponent.  

It has numerous big plays. The two teams should combine to have at least 14 “big plays.” A “big play” is one where the offense gains 20 or more yards, the defense gets a sack, a turnover occurs, a touchdown is scored, a safety occurs, or the team goes for a two-point conversion.  

It is captivating throughout. At no point in the game should there be an insurmountable lead, so the trailing team should never be four touchdowns (28 points) or more behind the leading team.

A “bad” game is one that satisfies any of these criteria:

It’s not competitive. The leading team is outscoring its opponent by three touchdowns (21 points) in the 4th quarter, and the opponent does not catch up to make it a one-possession game by the two-minute warning. Further:

It becomes a blowout. The leading team is up 28 or more points at any point in the game.

It’s low-scoring. The two teams combine for less than 25 points.

It has few big plays. Combined, the two teams have fewer than 10 “big plays” (see above for definition of a “big play”).

And finally, an “average” game is simply one that is neither good nor bad. “Average” games do slightly affect our definition of “bad”. “Average” games have some, but not all, qualities of a “good” game, and no qualities of a “bad” game, so we’ve separated them out from the final results.

Let’s go to some “deep ball” analytics.

Crunch Time with JQL

Now that we’ve defined “good”, “bad”, and “average”, we can run a query that shows the percentage of good, average, and bad games.  

Typically, when we want to calculate percentages of data belonging to different groups, the Segmentation report is ideal. If we already had a property called “Game Quality” for each game, we could answer the “good or bad” question with two clicks. But because our definitions of game quality rely on the values of multiple properties of multiple events, we must turn to a tool that enables us to operate within the data directly: JQL.

JavaScript Query Language (JQL) is one of Mixpanel’s most powerful tools. As the name implies, JQL enables you to answer any conceivable question you may have of your data by querying it in JavaScript, a modern and pervasive programming language. Think SQL, but more accessible.  

For the purposes of this analysis, we made use of the JQL Console, a web application that puts the power of JQL at your fingertips, right from the comfort of your own Mixpanel project.

console-1

Our JQL query followed these steps:

  1. Return all NFL plays from 2006 (when TNF premiered) to 2015
  2. Filter this data to only regular season games that occurred on Thursday, Sunday, or Monday, excluding plays recalled due to penalty
  3. On a per game basis, iterate through each play to check for the qualities of good, average, and bad games, determining the game’s quality
  4. On a per game day basis, iterate through the games and sum up the good, average, and bad games, while maintaining a total count of games
  5. Return the results in a useful format, displaying the count of games per day of the week, the percentage of those that are good, average, and bad, and finally the average number of big plays per game
  6. Rank the results from best to worst

Following those steps, we were able to quickly and simply construct a query on nearly 500,000 data points to confirm our hypothesis: Thursday Night Football is simply less competitive, and therefore less entertaining, than NFL football on any other day of the week.

results-1

With the lowest percentage of good games, the highest percentage of average and bad games, and the fewest big plays per game, Thursday Night Football comes in dead last. Sunday day games are tops: they have both the most “good” games and fewest “bad” games. (A quick data note: the total number of games has minimal impact on the data here, since we’re analyzing on a per game basis.)

But what’s this? Sunday Night Football is faring nearly as badly as Thursday Night Football using our gameplay metrics! Recall that SNF has had the highest ratings out of every other show on TV for three of the last five years. And while it’s true that SNF’s “good” and “bad” game percentages are saved from the cellar only by TNF, SNF fares better than all other game days in one major way: Sunday Night Football is #1 in “Big Plays” (20+ yard offensive gains, QB sacks, turnovers, touchdowns, safeties, two-point conversions).

The only thing missing in the data is the why of these fluctuations in gameplay. Why do Sunday Night Football games offer more of a spectacle? Maybe the players are fighting against the inevitable Monday morning like the rest of us. Or maybe SNF’s total prime-time dominance and its lofty branding as “FOOTBALL NIGHT IN AMERICA” makes it a prime example of a rising tide that lifts all boats (and gameplay). No matter the reason(s), TNF has a lot of catching up to do if it wants to compete with SNF

There is no substitute for good data… or good football

For decades, the only Thursday games were held on Thanksgiving. Not content with two back-to-back contests on our national day of feasting and football, in 2006 the NFL introduced Thursday Night Football, giving us football fans an extra game (or two – when Thursday Night Football occasionally shows up on Saturday), every week of the season.

But the data shows things didn’t change for the better. Thursday games themselves are bad on a football level, especially compared to Sunday games.

The gut feeling is no longer just a perception – it is a simple truth of twenty-first century gridiron football, backed by almost a decade’s worth of data. Maybe it’s that more points of entry for the football universe mean more chances for mediocrity. That’s just the law of averages.

Now that we’ve explained why your TNF experience is objectively worse than Sunday Night Football (or, really, any other NFL viewing experience), you can go find something else to do with your Thursday nights. Or not. After all, lackluster gameplay or not, TNF is still football. And those Color Rush uniforms are pretty cool. Sometimes.

tnf-1a-ctt

dots

Artwork courtesy of Jack Kurzenknabe, and is in the public domain.

Get the latest from Mixpanel
This field is required.