What is retention analysis?
Retention analysis is a method for helping product teams answer the question, “How many of our new users remain customers?” Also known as survival analysis, retention analysis is calculated over a period of time and serves as a vital supplement to new-user acquisition metrics because high retention helps to ensure profitable growth.
Why is retention analysis crucial?
Product teams never really know for certain what’s happening inside their app. Even with product analytics, it’s all too easy for teams to develop a false sense of security that users will keep using.
Many of these same product teams focus too much on acquisition metrics like new user growth because these numbers are exciting, easy to measure, and appear to be top-of-mind for everyone from executives to investors. But high growth paired with a lack of insight into how their app retains customers can conceal serious performance problems from product teams.
Product teams need retention analysis to understand how they can keep more of their users. If the marketing team is paying heavily to acquire new users who quickly churn, those users might have a customer lifetime value (LTV) that’s lower than their cost of acquisition. This means the company is losing money on the customers. Without acquisition spend, the app could not continue to survive.
Product teams can use retention analysis to answer questions such as:
- How long do most customers stick around?
- How long do various customer personas stick around?
- Have recent product changes helped or hurt retention?
- What changes are likely to improve retention?
By answering these questions, product teams can plug the holes in the leaky bucket they didn’t realize they had. It can help them:
- Develop a plan to increase retention
- Evaluate customer acquisition channels (PPC, affiliates)
- Make their app more profitable
How to conduct retention analysis
To calculate their retention rate, product teams must know three things:
In-app user behavior
Product teams must know what’s happening within their app. That means tracking users as individuals and as groups as they engage in activities like downloads, account creation, sign-ups, payments, downloads, shares and more. They must also track each user back to their acquisition source, such as an ad. Teams can then test to see which events and sources are correlated with retention.
Product analytics makes tracking easy. Learn more about Mixpanel.
Their retention definition
Next, product teams must define what retention means to them. Usually, this is specified as a number of activities within a set period of time, such as 10 days. But this varies from business to business. For a social media app, a user going dark for one week might be significant but with an enterprise accounting software, a one-month absence might not be noteworthy.
Here is a sample definition: A customer has been retained if they’ve taken any action within the previous 10 days. If they don’t take an action for 10 days, they are considered to have churned.
Finally, product teams need to divide their users into cohorts so they can compare how each fare. Typically, cohorts are defined by the week in which users created a new account. For example, each month would have four cohorts, one for each week. Product teams could compare January week one to January week two or February week one to see if retention has improved.
To easily track and compare cohorts, use an analytics platform like Mixpanel’s Cohorts which allows product teams to create, save, and reuse them anywhere throughout the platform.
For most apps, it’s any action that a user can take, such as logging in. Some companies may choose to only count actions they consider valuable, such as commenting or sharing.
With established definitions, product teams can take their retention measure. The most common form of retention measurement is retrospective retention analysis, which answers the question: “How many of our new users remain customers?” It visually displays how many members of each cohort were retained over a trailing period so product teams can see, for example, that after 30 days, they have retained only 10 percent of users.
From here, product teams can begin asking questions:
What is my retention?
Product teams can compute a variety of retention rates as they pertain to their business. For a consumer music app, that might mean calculating retention for all users at seven days, 14 days, and 30 days. For a marketing automation software with long contracts, that might mean calculating year-over-year.
Product teams can also calculate retention by each of their customer personas. When evaluated in conjunction with the LTV of each persona, the results can be illuminating. Some personas may demonstrate high potential but have low LTV because of their low retention numbers. By increasing their retention, teams can unlock much greater value.
And finally, product teams can evaluate retention based on acquisition sources. It’s not uncommon for certain sources to produce customers with either very high or very low retention rates. Ads, for example, might bring in a glut of new users who quickly lose interest. Social shares, on the other hand, might surface users who were dying for a solution and who are much more loyal.
How are our metrics trending?
Just as important as measuring retention is measuring it over time. Knowing that a product has a five percent 30-day retention rate is meaningless unless it’s compared to previous rates. If last month was six percent and the month before eight percent, product teams have a crisis on their hands. If it’s up from four percent, however, that’s cause for celebration.
Want to know the average retention rate for each industry? Download the 2017 Product Benchmarks Report.
Is there enough engagement behind our retention?
Product teams can use engagement to measure the quality of their retention. Consider that retention is typically measured based on a low bar of quality: any action within a product or app. If the users simply sign in and then out, they might qualify as ‘retained’ but they would appear to be equal to a user who logs in and spends three hours commenting on posts. By measuring engagement, product teams can understand whether or not their retention is hollow.
How do you measure engagement? Read about mobile analytics metrics.
What activities are correlated with retention?
By looking at their retrospective retention analysis, product teams can unearth ideas to better retain their users. For example, if they break the data down by cohorts, they might see that some are retained at a higher rate, and can see what mixture of actions or events sets them apart.
Product teams can repeat this process for all manner of cohorts, such as persona, acquisition source, product changes, and specific activities. They can isolate the variables that appear correlated with retention such as making friends, passing an onboarding quiz, creating a report, or making a purchase, and then focus their product design and marketing efforts on driving users to attain those early achievements.