The point of Bullshit metricsLast edited: Jan 23, 2021
_Re-posting of a byline that originally ran on Sufficiently Advanced, the personal blog of Mixpanel’s co-founder and CEO Suhail Doshi. _
I think we’re starting to miss the point of bullshit metrics.
Liz Gannes, from AllThingsD, recently called bullshit on LinkedIn reaching a new milestone: 200M registered users total. She made a great point: “The reality is, people can sign up for your site and never return, or become frustrated and quit, or just stop caring at some point.”
Josh Elman, from Greylock Partners, recently wrote a post defending LinkedInÂ’s metric claiming that “for LinkedIn, I believe that what matters for the size of their business is exactly the number of registered users.” He also claimed that “the only thing that matters is how many people are available to be contacted and are likely enough to respond.” My first instinct was to pull LinkedIn’s quarterly earnings reports because I knew they likely segmented their revenue into at least 3 different categories that were all significant. Next, I was going to show how the number of registered users didn’t logically correlate to many of its largest revenue lines. In the end, I felt that just wasn’t the point of bullshit metrics.
Sarah Lacy, from PandoDaily, also wrote a post talking about how she wasn’t buying the idea of one key metric. After all, how could a company run their entire business off a single metric! That seems dangerous! She did at least admit that “page views are clearly a bad single barometer for any news organization that wants to prioritize quality.”
After reading about people’s varying opinions, my epiphany was that we’re all on the same side! We all want the same thing for the world: to start reporting and measuring metrics that matter and make a real difference to businesses. We want companies to talk about metrics that are harsher than the overused total number of registered users.
I feel like weÂ’re missing the big picture by debating the smallest details of whether a metric correlates heavily or not. The problem with that is that every metric is contentious and debatable in some fashion. You cannot always control for weather, seasonality, time, and historic events. ThereÂ’s always a reason something isn’t necessarily what it appears to represent. And it’s reasonable to debate that, when it’s important, but the point of talking about bullshit metrics is to end the use of poor metrics like pageviews, the number of downloads, and the number of registered users ever. It’s to get companies to focus on data that likely represents a conclusion they can make about their business. And it’s to have the world represent themselves in a way that doesnÂ’t perpetuate a constant cycle of distorting reality. Measuring a customer’s engagement and retention are just the beginning of a mainstream solution to this problem.
LinkedIn has an incredible dataset (that I hear they can use to predict macro- economic trends) and super smart data analysts. Presumably, they can tout something far more substantial and representative of LinkedInÂ’s actual growth than the total number of users that have ever signed up.