In several months, Facebook’s event, f8, is going to launch its third
hackathon. Every year I like to take a step back and ask, “what did I learn?”
With Mixpanel, we have the opportunity to work with
lots of companies and developers, helping them really understand what their
analytics mean with regard to their applications. I am going to go over some
of the general things that I think all developers can benefit from:
Also called “cohort analysis,” understanding visitor retention is probably the
most underrated yet most important metric all Facebook developers should be
looking at. Visitor retention helps you understand how “sticky” your
application is by being able to tell you the percent of users that came back
to your application 1-6 weeks later.
In the early days of Facebook’s platform, lots of the applications with the
highest amount of installs have now failed because their retention was
extremely low. The only way to increase this is by giving really good reasons
to users to come back again and use your application. Sadly, applications like
Hug Me were likely only fun the first few uses.
Max Levchin, CEO of Slide — my former employer — has described what happened
with applications early on as large hollow spheres where recently acquired
“users” made up the outside. This was in part due the sheer number of users
inviting each other and accepting invites/notifications. The ideal app is a
sphere filled with users acquired previously as well-meaning returning users
combined with new users which make up the outside.
This breaks down as follows:
< 20% (week over week) is poor and needs serious improvement.
< 35% (week over week) is pretty good > 45-65% (week over week) is excellent
The reason games on Facebook really took off was large in part due to great
retention in comparison to competing top applications. Games provided really
great game mechanics and incentives to come back. The concept of “farming,”
for example, has lots of replay value.
With retention, we generally see that really good applications have fairly
consistent week over week retention. This means even 3 or 4 weeks later the
retention is roughly the same as the number of users who came back just 1 week
Even if you have great retention, if your users don’t convert well, it’s going
to be tough to grow. Funnel Analysis will tell you what percent of users go
from step 1 to step 2 to step 3 (so on and so forth).
1. Intro video with a story
2. Daily Bonus (Earned $100!) telling you to play everyday for more
3. Chapter 1 introduction
4. Big “Click here to get started” call to action on a map with a location.
5. “What’s your name?” input box
6. Pick a mission to do
7. Accept mission
8. Cash earnout / More jobs page
9. Share the mission you won (publisher)
10. If you do another mission (which is what most people will do), you level
Difficult spots for most funnels will be between step 1 and 2 and Playfish
immediately gives the user $100 the moment you enter the game to convince and
incentivize the user to proceed.
It’s really important to make sure your funnel is converting well.
Interestingly enough, with most Facebook applications that are games, we
generally notice that after step 2, users tend to convert over 90% in
subsequent steps (we’ve seen it up to 7+ steps). This is one reason why lots
of games have tutorials.
In Gangster City the subsequent steps really just introduce the user to the
“farming” mechanic: missions. Since this is fairly mechanical, it’s likely
they convert at the 90%+ rate most game flows do convert at.
Lots of games right now use the funnel to help track how much their users are
spending money in games, what that conversion rate is, and then use that
information to figure out how much they can spend on ads to funnel in more
users. This isn’t easy to calculate — most hardcore metrics types will talk
about the lifetime value of a user and how invites and notifications can
discount the price it takes to purchase a user.
Good applications on Facebook should see funnel completion conversion rates
above 65% even if those games make users purchase items and invite friends as
part of the flow.
For more information about funnel analysis, check out our blog post “Introduction to
Products that have really great retention have the best chance at success.
It’s usually much easier to adjust, experiment, and play with your funnel to
optimize your conversion. A product that has very bad retention in the long-
term will most definitely fail. Now, extreme viral growth seems less important
in the beginning on Facebook because of the hooks it provides to gain
distribution. You don’t want to waste time growing a huge application (like in
the early days) and find out a year later that it is failing due to low
The best way towards building a long-term sustainable product from companies
I’ve seen is to first focus on achieving a decent portion of users (10k-50k).
Second, focus on figuring out how to achieve high retention. Lastly,
optimizing your funnel while focusing on viral mechanics for growth.