Less, but better: how focusing on UX helped us triple NPS in 18 months

Sometimes, the simplest path is the best path forward.

With a renewed focus on our core user experience, we saw our NPS score triple in the span of 18 months—from 15 to 50—and we’re already feeling the impact of these changes across the company.  

How?

A relentless prioritization of UX-focused, product-led growth. 

Though the UX initiative has been just a small piece of Mixpanel’s product-led transformation story, it’s been a surprisingly impactful one, surpassing even our own expectations. 

But success doesn’t come without challenges, and we encountered our fair share of bumps in the road. In the paragraphs below, I’m sharing the lessons our team learned along the way, with the hopes you can apply some of these learnings too. 

⁉️ Too often, product analytics fails to deliver on its promise

Three years ago, I started at Mixpanel as a PM focused on two of our core products, Funnels and Flows. Along with other Analysis PMs, I was very happy churning out new features at almost factory-like speed to satisfy customers. 

The goal was to help us win deals and close gaps. While I’d love to say otherwise, not every single feature resulted in a better overall user experience. In fact, from the angle of a new user, the product was getting more complex and harder to adopt for everyday use. 

This isn’t to say we didn’t do our best to follow the data.

Along the way, we conducted in-depth research on competitive deals, talking to customers who churned from us, or from our competitors. The hard truth is that as a rapidly evolving category, product analytics is, well, hard. New customer sign-ups often come down to feature depth and custom use cases, while churned customers tell a different story: losing trust in the data, or seeing low adoption in their team. 

In a nutshell, product analytics has grown incredibly complex at the cost of everyday users.

💔 The NPS score that broke our designer’s heart 

Our NPS feedback was telling us the same story. 

Before our renewed focus on UX, our NPS score sat at 15. (For context, SaaS sees an average NPS score of 30, so we knew there was room for improvement.) 

Taking a look at our own NPS data, we found out that 50 percent of customers’ NPS comments were linked to UX. Looking at this data in more depth, we realized that our—let’s face it—relatively low score was a story of 1,000 UX paper cuts.

Some of the low scores were driven by the complexity of product analytics and the challenges users face as they learn their data lexicon. But, frankly, other scores were soul-crushing pieces of frank feedback that gave us a wake-up call, broke many of our designers’ hearts, and showed us the potential for improving our product experience. 

If you’ll indulge me in a meme, the whole experience can be summed up by this:

Our users are incredibly excited about the promise of product analytics, but the complexity of our UX was preventing them from discovering ‘aha’ moments, and building habits. 

Given all of this context, we started asking ourselves: is it possible to change our UX as a company with 10 years of industry experience and an intricate product? 

🗺️ Product strategy reset: Less, but better

Inspired by these words, our product team started to focus on the core product analytics workflows in addition to meeting our customers’ ever-evolving demands. 

We formed an internal task force of a few PMs, designers, and engineers to double down on improving the usability of Mixpanel to turn our product perception around.

I was very lucky to have a chance to guide this team.

First things first: focus on the right metric

In the past, when we were looking to address churn as a company, we looked primarily at retention and the NPS score itself as our core metrics. But we learned the hard way that these efforts do not lead to the desired outcomes.

Instead, as Hubspot’s Dharmesh Shah points out, aligning input metrics to flow through to each other will have the maximum impact on your output metric. 

In a phrase, NPS is an output metric, not an input metric. While our goal was to improve NPS and retention, we needed a better focus metric.

Defining Mixpanel’s focus metric

After months of analyzing user data, collaborating across teams, and researching outside sources, we doubled down on the focus metric that captures the essence of Mixpanel’s value proposition. 

We found that users love our product for the simplicity of asking follow-up questions, and slicing and dicing data in just a few seconds to learn new insights. This is what we call segmentation within the product: when users apply a filter or break down in their queries, which leads to getting an answer to specific questions in Mixpanel. 

With that in mind, we defined our focus metric as Learners 3d7: users who go into Mixpanel 3 times per week to segment data as they ask specific questions and seek new insights. Unsurprisingly, this group retains at 3 times the rate of our baseline user retention. 

With more than a little bit of whimsy, we called this new focus metric LUV: Learning User Verified. 

(If you’d like to replicate this analysis for your product, you can find more detail in our blog post on the topic here.)

🌈 The discovery of a new user’s “happy path”

With our focus metric at the forefront, we started our search for what we called the “happy path” of new users, and codified a funnel that maximized the rate of those new users activating & falling in “LUV.”

The key, we realized, was to focus on new users and not recurring users, or existing power users. 

Why? Because looking at long-term returning users would lead to major survivorship bias (especially for Mixpanel, a product that’s been around for 10+ years). 

For every retained customer who was activated via a particular path, there could be 100 customers who failed to activate on the same path—but you’d no longer see them in the last snapshot of data, as these users might’ve churned long ago. 

When we focused our search on new users and evaluated hundreds of different paths, we were able to find the most optimal one. 

The optimized path for user activation

After getting an in-depth understanding of the paths our new users took, we came out with an optimized one that consists of 6 steps (that now, to be honest, seems very obvious). 

Users who completed this funnel retained at 4x the rate than the baseline. 

It consist of these steps:

1. Sign-up: A user successfully signs up (this is sometimes overlooked, but if you ask a user to fill out a 10-step form just to take a peek at the product, you won’t get that many users). 
2. Content discovery via dashboard: A user finds a useful dashboard built by their colleague that answers their questions—and gives them a reason to keep coming back to it every week. 
3. Viewed core report: A user digs deeper into a particular report to understand what events define a given metric, and how it behaves over time.
4. Learning action (i.e. slicing and dicing data): A week goes by and the user looks at the metric in detail, segmenting the data to find out why it’s changing and what segments are driving the change. 
5. Value moment: The user saves this analysis, adding it to the team’s dashboard. This closes the content loop and increases the likelihood of future users jumping onto the path to Learning User Verified action. 
6. Sharing: The user shares it with a new user who is then invited to Mixpanel, closing the social loop and adding a touch of virality to this flow. 

At the time we codified this “happy path”, the conversion rate was far from perfect—but it gave us clarity on what to fix. It led us to double down on a handful of projects, outlined briefly below. 

🚢 Snapshot of the UX-focused projects we’ve shipped 

I. Shipped: New user onboarding

We wanted to make getting into the core Mixpanel product as simple as possible, teaching users core aspects of the product (users, events & profiles) to set them up for success in the product.

II. Shipped: Dashboards and content discovery

Before:

After:

We revamped our dashboards and content discovery features by:

• Simplifying the top nav, and limiting it to Dashboards, Reports, and Users. We also introduced a new search and Library experience with dashboard/report view counts to quickly discover a report you can trust.
• We added a side nav, which lists out “owned”, “pinned”, and “favorited” dashboards for individual users. 
• We added new charts to show additional insights right on the dashboard, including:
  • Revamped Funnels segmented chart
  • Retention line chart
  • Compare to Past Insights bar chart and more

III. Shipped: Report query builders

Before:

After:

One of the core features of Mixpanel is the ability for users to query data for customer insights, flows, funnels, and retention. But, until our UX update project, these query builders weren’t consistent—each approach required a different set of fields, so each builder looked different. We made the UI similar across each, and added component “sections” for clarity. 

IV. Shipped: Many others

Altogether, we shipped 92 product upgrades designed to improve users’ learning path. 

These updates included things like:

• Report descriptions in dashboards
• Working links from dashboards
• New “explore users” table
• Revamped date-picker UI
• Slack app integration for sharing

🎉 The results

In the 18 months we worked on this, we saw three things happen:

• Our NPS score tripled, from 15 to 50.
• We saw a triple-digit improvement (YoY) in our activation funnel. 
• We also experienced a notable increase in word-of-mouth and positive customer sentiment (via social media, G2, etc.)

While our focus metric was increasing the rate of users who hit LUV, we’re excited about what these changes mean for customer satisfaction, adoption rates, and word-of-mouth growth.

The takeaway? A small change in NPS score can drive your product’s stickiness and viral growth.

Wise (formerly TransferWise) did a study on the impact of this strategy for their own digital presence, and found that even a small increase in your NPS score can double the number of customers who tell others about the product. Moving a customer from an NPS of 5 to an NPS of 8 doubled the number of people they told about the service, and moving a customer from an NPS of 8 to an NPS of 9 doubled their word of mouth again. 

📖 What can you gain from a UX-focused, product-led strategy?

I’ll be honest: while we knew focusing on our UX would improve activation and retention, we didn’t understand its full potential from the outset.

Now we do. 

More than anything, we realized that focusing on UX can be your secret sauce—because it truly touches everything:

• Improves satisfaction with day-to-day use, impacts growth, and drives word-of-mouth. 
• Impacts retention (i.e. if you have a lot of daily or weekly users, your customers are very unlikely to churn). 
• Enables a shift to product-led growth, in which users are engaging from day one, and your Sales team can talk to excited users who have already warmed up to (or even fallen in love with) your product.

Of course, what works for one business might not work for another. But for the reasons above, I strongly advise that product teams take a closer look at how small improvements to UX can increase NPS scores, activation rates, and overall retention.

Above all else, I’m looking forward to seeing more PMs drive this transformation for the benefit of customers everywhere. 

Looking to learn more about product analytics in action? Check out our guide to product analytics put together by 25+ product leaders.