Magic numbers are an illusion

“Seven friends in 10 days.” It was Facebook’s sole focus in its growth from zero to one billion users. I hear it all the time when I’m talking with product people. And now everyone wants to find a magic number like Facebook’s. Even if they aren’t sure what it’s called, they want it, or some approximation of it. Because who doesn’t want to grow to a billion users?

Magic numbers like “seven friends in 10 days” are basically a take on the “aha moment.” They are that wondrous tipping point where a person understands the value of a product and is transformed into a lifelong user.

But if you go digging into your data looking for magic, you probably aren’t going to see it. That’s because magic numbers are an illusion. A very useful illusion, but an illusion nevertheless.

If you’re looking for just one thing to indicate whether a user will be retained, you’ll be hard pressed to find it. And that’s okay.

With “seven friends in 10 days,” Facebook knew getting a new user to add friends and begin building out their social network would keep that person coming back again and again. And so that magic number became their sole growth focus. Of course other companies, like Twitter and Slack, have followed suit with their own magic numbers.

Not surprisingly, many startups long for the adoption rate and growth of Facebook. And so they want that magic number or that a-ha moment. They want to activate and retain new users, since that’s how to sustainably grow a user base. And the big guys made it seem so simple. I mean, “magic” is right there in the phrase.

But then when the up-and-coming startups look at their data, it’s not really there. They get frustrated. No matter how much they slice up the data, there’s not one magical point where everything just comes together. Sure, different user actions lead to different conversion rates, but they struggle to find one magical tipping point when a person becomes a lifelong user.

As anyone who works with statistics will tell you, the last time you’ll see clean data is in a textbook. In the real world, it’s messy. Users who do Action A are X more likely to become an active user, and users who do Action B are Y less likely, but there is no one thing.

There really isn’t anything inherently magical in a so-called magic number – even for Facebook and their often repeated “seven friends in 10 days.” In all likelihood, as Andrew Chen pointed out in a Quora post for those in search of a magic number, it could have been “10 friends in 12 days” or even “five friends in one day.” That’s because magic numbers aren’t entirely the output of some intense mathematical number crunching. Absolute precision into what makes a person become a lifelong user isn’t possible.

Instead, magic numbers are about finding when users get value from your product and working like hell to get them to that point. That’s when they’ll start coming back again and again.

With Twitter, that happens when you follow 30 people.

“Once a user follows 30 people, they’re more or less active forever,” Josh Elman, who worked growth at Twitter and is now a partner at Greylock Partners has said. “If Twitter couldn’t get a person to follow 30 other people, that person was very unlikely to ever come back.”

And that metric influenced how Twitter built their onboarding flow. As a part of account creation, new Twitter users select suggested users to follow. This way, when they see their Twitter timeline for the first time, it isn’t a ghost town. It’s populated with tweets from celebrities, athletes, and people they are interested in, to give them a feeling of what the Twitter experience is, right out of the gate.

The goal was to get new users to that magic number, so that they could understand the value of Twitter, as quickly as possible. Even a more ephemeral product like Slack has its aha moment. It is trying to get your team to send 2,000 messages. These companies have found these magic numbers and focused their efforts on getting new users over those number.

These Magic numbers are more like mantras, shorthand for growing companies looking to prioritize their efforts.  They allow different teams, working in different departments, to rally around a simple phrase and work towards a broad company goal. In Facebook’s case, it’s about seeing beyond new user signup to integrating with those users’ personal network of friends and family as quickly as possible. Then each team can apply that to whatever they were working on, whether it was the UX of the onboarding or the suggested friends algorithm. It’s not an exact GPS location, but it is a north star.

“What you value is really what you achieve,” Chamath Palihapitiya, former product growth at Facebook tells a room at the Growth HackerS Conference.

For most people, that would be aspirational. Chamath is not most people. Instead, he’s imploring a room full of growth hackers not to get caught up in their own world of ancillary metrics like k-factor and to focus on delivering value to users.

“I’m raining on everyone’s parade today. Core product value is really elusive, and most products don’t have any. I actually fundamentally believe that. But I also believe that most products can have some value.”

Palihapitiya was a big factor in Facebook’s climb to a billion users and the “seven friends in 10 days” mantra. And even though he’s moved on to being CEO of Social Capital and owner of the Golden State Warriors, everyone still just wants to talk about Facebook’s growth.

“It’s like, ‘What’s the secret?” Almost as if we’re the NSA and and we’ve developed something nobody knows, like we’re in some secret backroom negotiations with the government. It’s none of that shit.

“Get any individual to seven friends in 10 days. That was it. You want a keystone? That was our keystone. There’s not much more complexity than that. Now there’s an entire team, hundreds of people, that have helped ramp this product to a billion users based on that one simple rule. So if you were looking for a lot of complexity I couldn’t give it to you.”

It’s easy to get lost in the complexity, especially when trying to find your own magic number. Maybe a new Facebook user actually has a higher likelihood to convert to a lifelong user if they add seven friends in 122 hours, log in five times, post two status updates, and “like” three posts. Maybe that’s actually a metric more indicative, but who is going to remember that? Sometimes it’s just about keeping it your mantras simple and memorable.

You can mine thoughtful insights from the data, but if you can’t package those insights in a way that gets the rest of your team onboard, and acting on them, then what good are they?

Sam Miller is the editor in chief at Baseball Prospectus, a yearly tome of advanced analytical breakdowns of each and every player and team in Major League Baseball. It’s a bible for “Moneyball” number crunchers who obsess over PECOTA, VORP, and FRAA.

He knows a thing or two about parsing data for insights. As does his friend Ben Lindbergh, a writer at the ESPN-run statistical analysis website, FiveThirtyEight. And for years, the two have pontificated over how exactly the game should be played, all backed up with mountains of data.

But last year they got an opportunity to put that to practice when the Sonoma Stompers of the Pacific Association brought the pair on to run baseball-operations – a task that includes building the roster and making decisions about on-field strategy.

And at first, things seemed to be going to plan. They were quite successful, winning the most games in the league in the first half of the season. But then things started to falter.

“In the second half, when we should have been leveraging our data, our team began to collapse. The more data we had, the worse we played,” Miller wrote in the NY Times.

“How had we gotten this so wrong?” he wondered.

Arguably, Miller and Lindberg are as qualified to make informed baseball decisions as anyone on this planet. But something was wrong. The team wasn’t taking action on their data-driven insights. It wasn’t sinking in and affecting their play.

“We sold ourselves as something imposing – data analytics – and we made it about us. We should have sold it as providing them information and made it about them.”

Ben and Max knew they had to repackage their data-driven insights into something that would resonate with their players.

“With other sabermetricians, more data wins arguments. In the dugout, a good story does.”

For years, number crunchers have written stats-fueled screeds about why a team shouldn’t wait to use their closer  – a dominant pitcher that only pitches one inning  – until the ninth and final inning.

“When we started using our closer in tight spots as early as the fifth inning — instead of the ninth, as every other team does — we kept our message as simple as could be: The game is on the line, so let’s take the bad pitcher out and put the good one in.”

They sold the simple story. There were numbers behind it, but they packaged it into what they knew would play in the dugout.

It’s simple, but that’s what a magic number is at a company. It’s a one-sentence story that resonates with your company and gets everyone onboard towards the larger goal.

So how do you tell that story? How can you create your magic number? First, take a good look at your product and your user. What do they do when they first begin using the product?  Look at those early events in a user’s timeline that affect retention. And make note of the events where a user starts to understand the value of your product.

With all that in mind, any company can look at their retention numbers, understand where the friction lies and when users get value from their product, and create a story around a magic number.

A few weeks ago I sat down and talked with the folks at VSCO about how they buckled down and got serious about being data-driven. In an Office Hours talk,  Matt Turner, Product Manager and Steven Tang, iOS engineer, shared some outcomes of that focus, including a few of their very own feature-specific magic numbers.

VSCO is an app for people to create, share, and discover photography. And like many products that have grown over several years and many iterations, VSCO is a robust app with quite a few features. And that means there isn’t just one way to be a user of VSCO. So when a person opens up VSCO for the first time, there’s a few different ways that they could use it. Maybe they are just editing their own photos. They could be publishing photos to their account. Or they could just be looking to find great pieces of photography and save them in a collection. Maybe they do one or two, but to the get the most value from the product, VSCO wants to get users to do all three.

But using a feature one time is not enough. VSCO wanted to make people into regular users of its different features. But what does that mean, and how can they create the story to rally the whole company around?

The first problem was that all user actions are not created equal. Some actions are stickier than others. So the VSCO team looked at each feature and figured out how many times a person had to use it before they kept coming back to it again and again.

It turned out, editing photos was the stickiest. It took editing eight photos before users hit the 90% retention that VSCO was aiming for. Publishing an image was a bit less sticky, meaning users had to publish 10 times before they hit the retention goal. And, not surprisingly, collecting images was the least sticky. It took 16 collected images before users were retained on the feature.

With that in mind, and knowing that people would find VSCO more useful and would be less likely to churn if they used multiple features, the goal became “How do we get people to edit eight photos, publish 10, and collect 16.” That would make a person a VSCO power user. They called each a “user milestone,” and worked to get people from one milestone to two to three.

It’s informed by data, but the story is simple. Just get users to another milestone. One may come before the other. One person may be a “collector” and so you try to get him to be an editor. Or she may be an editor and publisher, but you’re trying to get her to be a collector. Whatever the path, each milestone was really about  how to return more value to the user.

And, of course, VSCO’s data team could have put the feature retention rate at 80% or at 95%. They decided on 90% because it was simple and memorable. The result wasn’t an exact formula, but it’s informed by their product knowledge, user data, and condensed into a nice, memorable goal that anyone internally can harken back to.

So a magic number is, to a large extent, an illusion. But a very useful one. There’s some soft science behind it, teamed with a lot of product and user understanding, a little bit of hand waving, all packaged into a simple and memorable story.

Because, ultimately when it comes to something like Facebook’s “seven friends in 10 days,” it’s all about getting buy in on what makes your users love your product. For all the tips and tactics, great products are about creating something that people will get value from. That’s how you grow a product from zero to a billion users. And if a good story and a little false magic gets you there, then by all means, use the illusion.

Photographs are by Håkan Dahlström and popofatticus, and are made available under an Attribution 2.0 Generic license