How OneLaunch accelerates innovation through an A/B testing culture with Mixpanel

• Improved conversion during the onboarding process by 60%
• Discovered a significant opportunity to monetize page views within the OneLaunch browser

How they did it

Prior to Mixpanel, what were the problems and challenges you were experiencing?

We actually built our own tool for product analytics, which we called ‘Query Builder’. It worked to an extent, but we were never 100% sure it was pulling from the correct data. The biggest issues were functionality and the user interface.

We tried to build an engagement funnel and several other capabilities but these didn’t work as we desired, so what we could actually track and measure was really quite limited. Data visualisations were also very basic, and we couldn’t really interpret the data well, which led to our team only using the tool when we absolutely had to. We weren’t really exploring our data well. Ultimately, this meant we didn’t understand how our people were using OneLaunch, and we tended to hear about problems directly from our customers or our partners.

What issues resulted from these problems?

We had a challenging time understanding what our users were doing within the product. So despite having lots of data, we couldn’t answer even simple questions like where our users were falling out of our onboarding funnel. When we launched new improvements to OneLaunch, we had little insight into whether or not they were well-received or generally what impact they had. This was a real issue for us because OneLaunch is all about improving the experience of using a PC. We really need to understand whether each enhancement we make is contributing to a better Windows experience and have the confidence to release those improvements.

Why did you choose Mixpanel?

We heard about Mixpanel from a number of people in the industry that we really trust. These people are computer science and product experts, so you know if they’re happy with a tool then it definitely has something to offer. We also conducted a thorough due diligence process and were very satisfied with Mixpanel’s power and how simple it was to use.

Tell us more about Mixpanel’s ease of use.

We never thought we’d say this about an analytics tool, but Mixpanel is actually a fun tool to use! It’s intuitive and very simple to navigate. The insights can be shared easily, and we regularly drop links to Boards in internal Slack channels. It’s unlike any analytics software we’ve ever used before, and it’s an incredibly powerful tool that allows you to get very granular and to run a wide variety of analyses.

How did you drive adoption of Mixpanel within your organization?

Every team at OneLaunch uses Mixpanel. Product uses it to understand user behavior, to test hypotheses, and to create reports, cohorts, boards and reports on key metrics. Engineering, marketing, and leadership consume Mixpanel insights in the course of their daily work.

Originally we shared onboarding materials that were helpful, but true adoption has really been very organic. From sharing reports in Slack, colleagues have become curious; they see how the insights can help them and they want more. Our team quickly saw how powerful Mixpanel was and were naturally drawn to explore its full capabilities. Now everyone is a regular user of Mixpanel, and even non-technical users are beginning to ask questions about our data. We’re definitely far more data-informed since onboarding Mixpanel.

It also helped to have a strong champion within our product team who made a point to use Mixpanel exclusively. This makes a big difference when your organization has a champion who leads by example.

Has Mixpanel helped you understand what is and isn’t working within your product?

Yes- we run a lot of A/B tests- also called split tests- to learn how our users may respond to new features we release within the product. We examine the impact the new features have on different install cohorts against a control group of users. This helps us compare the behavior of different cohorts based on who did or didn’t get the new product offerings. For example, OneLaunch will sometimes prompt users to take an action we desire, but it’s a very delicate balance. We don’t want to interrupt the user’s experience on their PC, and we need to understand which type of user should be prompted, how frequently, and which prompts add value. Using Mixpanel alongside A/B testing helps us achieve this balance.

How has Mixpanel helped you drive business value?

Perhaps the clearest example is funnel retention improvements. By mapping our onboarding funnel, we were able to monitor the user flow, identifying points of friction that were causing users to drop-off during our ‘install process’. We learned that when users first executed OneLaunch after download, they were presented with an intimidating disclosure window and that was the issue. Mixpanel gave us the insight and confidence to rework our disclosures and to spread them earlier into the onboarding flow.

By addressing those friction points, we’ve been able to improve our onboarding conversion rate by 60%, which is huge for us. This is a great example of direct business value, and it’s also great for our users. We’ve been able to reinvest the resulting additional revenue into user acquisition marketing. And with more users completing the onboarding flow, we’re seeing greater Return on Advertising Spend from those campaigns. It’s been a virtuous circle that’s supercharged our performance.

Mixpanel has also helped us reduce time-to-insight significantly. We can see technical metrics like ‘version performance’, as well as retention, engagement, growth and install rates, which marketing, leadership and product colleagues all value.

You’ve discovered an entirely new business using Mixpanel, can you elaborate?

OneLaunch is a browser, and we have a screen that loads when people create a new tab. Historically we hadn’t given the new tab screen too much thought, but by using Mixpanel we discovered we had an enormous number of monthly impressions on that page. From that discovery, we started to consider how we could add value to our users. We added weather information and news, and the audience continued to grow–in fact it doubled. This actually turned into an entirely new business for us as we began to gradually monetize the page with ads. This has had a substantial impact on topline revenue, especially as it is “free” traffic from within our browser. Without Mixpanel, we wouldn’t have discovered this opportunity or built this value.

What else have you discovered using Mixpanel?

Sometimes less can be more. Having too many features is not always a good thing. We’ve simplified and cleaned our product by removing features that our analyses showed were not valued by our users, and that’s improved the overall OneLaunch experience.

Another surprise has been the impact of seasonality on our user base. We had no idea that usage of a generalist product like ours would be impacted so significantly by holidays. Understanding seasonality helps us know when to rollout new updates and when to minimize change.

In closing, what would you not have been able to do without Mixpanel?

Everything would have been way more painful. We would have less exploration and less wins. We just celebrated a big milestone- 1 million daily active users! Mixpanel was integral in enabling us to reach that new DAU milestone. We’re grateful for our partnership and are looking forward to driving more success for our business and our users with Mixpanel insights.

Thousands of companies use Mixpanel to build better products. See what Mixpanel can do for your team. Create your free account today.