How user analytics reduces mobile app churn

Clement Kao is a Product Manager at Blend, a San Francisco-based startup that partners with lenders and technology providers to re-imagine consumer finance. He is also Product Manager-in-Residence at Product Manager HQ (PMHQ), where he’s published 50+ best practice articles, provides advice in the PMHQ Slack community (7,400+ members), and curates the PMHQ newsletter (25,000+ subscribers). In this guest post for Mixpanel, he shares his lessons on reducing mobile app churn with user analytics.

Product managers know that understanding user behavior can be powerful. After all, how can you drive value for your users (and your business) if you don’t know how people interact with your product? Yet often much of what we know about user engagement and customer loyalty is bimodal — users are either engaged or not, loyal or churned — and relying on such a limited view can lead us down the wrong path as we try to convert disengaged users into product champions.

Using quantitative user analytics and qualitative feedback enables product managers to understand the root causes of disengagement and churn, and to drive product changes that unlock significant business value.

In my years of product management and business consulting, I’ve learned that users become disengaged for many different, sometimes surprising reasons. Relying just on common sense and intuition can leave you blindsided. To illustrate, I’ll share a case study in which identifying trends in user behavior and understanding “the why” helped us supercharge user engagement and reduce churn.

The challenge: reduce churn for a mobile app

A few years ago, I was working as a PM on a CRM mobile app for real estate agents. The app enabled real estate agents to capture and follow up with leads coming from the website.

Our goal was to become an indispensable tool for all real estate agents. We wanted agents to stay active in the app, and do a better job selling real estate; our business model depended on it. We received a cut of the agents’ commission as well as an annual subscription fee. The more productive agents on the app, the healthier our business.

But our churn was high. Many agents would register, log in a few times, and then leave for good. Worse, these disengaged agents would spread negative word-of-mouth, which then hindered our ability to attract new users.

To tackle the problem, we turned to user analytics.

Identifying user behavior patterns that lead to churn

It was clear the lack of metrics wasn’t the problem. We had plenty of metrics — and still couldn’t understand where the churn was coming from. Analyzing overall user engagement wasn’t helpful either. Our previous approach included experiments targeting every user in the app. They didn’t work.

The inspiration came from field research. I was shadowing several agents and noticed that the most engaged users interacted with the app very differently than the disengaged cohort. I decided to segment the user base by engagement levels to pinpoint the specific events and actions that would indicate our most and least engaged users. 

We began by defining an “engaged” user vs. a “disengaged” user. Looking across the entire user base, we found that the most active quartile had a median login rate of 31.7 times a month, whereas the least active quartile had a median login rate of 4.9 times per month. Given how starkly different those quartiles were, we decided to use the top quartile as “highly engaged” and the bottom quartile as “highly disengaged.”

We then compared the user behavior of highly engaged users to highly disengaged users. Looking at the last 30 days, we found that highly engaged users regularly reassigned leads to teammates, rejected leads, and set vacations within the app. We found that highly disengaged users were far more likely to give in-app feedback.

The surprising actions that defined our power users

We were stunned by the results. We thought that the most engaged (and successful) agents would hold onto as many leads as possible, use our prioritization feature to order their pipeline by priority, and rarely take vacations. The data proved these assumptions completely wrong.

In reality, our most engaged agents rarely, if ever, used our prioritization feature to order their pipeline by priority. Instead, they frequently rejected leads. They also often reassigned leads to other team members. Finally, highly engaged users would regularly set vacations in the app.

Understanding what makes an engaged, happy mobile app user

Confused by these results, we turned to qualitative user research to find the root cause behind the behaviors we saw in our analysis. Through 1:1 interviews with highly engaged and highly disengaged users, we unearthed the following insights:

• Highly engaged users often rejected leads to focus on delivering the best possible experience for the clients that they did accept. They used the “archive” feature so they could focus on the active pipeline, and “unarchived” leads that became active again.
• Highly engaged users often reassigned leads to teammates who specialized in a specific niche, such as first-time home owners or repeat investors. Unlike disengaged users, who would take any lead from the website, engaged users also established specific niches for themselves.
• Highly engaged users didn’t use our “set lead priority” and “sort pipeline by priority” features (which we promoted as our app’s differentiating functionality). When asked why, the agents said their priorities changed frequently, and it took too much time and energy to manually reprioritize leads every time.
• Highly engaged users regularly set vacations in the app, which automatically rejected all inbound leads while they were on vacation. That helped them provide a reliably fast response when they were working.

We also uncovered why disengaged users were frequently frustrated about the app’s support. Our in-app feedback routed users to the product and engineering team, whereas our support hotline routed to a live support team. Our disengaged agents often submitted in-app feedback, and it took weeks before they would get a response. In contrast, engaged agents called the support hotline when they ran into trouble, and got immediate help.

Fixing UX to turn disengaged users into product champions

Armed with these insights, we came up with a few user experience changes to unlock value for our disengaged users. Here’s what we did:

• Instead of hiding the “set vacation” functionality behind multiple screens, we brought it to the very forefront of the pipeline and reduced the number of taps required to set a vacation.
• We increased the visibility of the “reject” button whenever an agent received a lead, and introduced additional copy in the app to call out the benefits of rejecting leads.
• We introduced a new “archived” view of the pipeline so that agents could quickly find archived leads. We also made it easier for them to unarchive leads directly from that view rather than needing to tap into the record to unarchive the lead.
• We launched an experiment around algorithmically determining the priority of leads, rather than having agents manually set the priority on every single lead.
• We removed the ability to provide in-app feedback. We replaced it with our support hotline, which they could call with a single tap.

Results: increased number of user logins and a boost in profits

After we identified the problems and acted on them, we were able to shift more people towards the ideal behavior of engaged users. Our median logins rose from 10.6 times per month to 18.3 times per month.

Increased logins drove a reduced churn rate and an increased referral rate, boosting both top-line revenue and bottom-line profits. We no longer had to spend as much in operational costs to prevent users from churning.

Key takeaway

User engagement looks different for every business. What I found about engaged and disengaged users in a CRM app for real estate agents was very different from my hypotheses. As product managers, when we try to improve our products solely by evaluating competition or relying on our intuition, we frequently make the wrong calls. (In my situation, the prioritization feature that we thought made us different from competitors turned out to be a flop.)  Digging into data, identifying behavioral differences across segments, and conducting in-person interviews helped us fix user experience in ways that majorly benefited our users — and our business.