Frequently Asked Queries - Chapter 03 - Mixpanel
Depth of analysis

Should I segment my product data?

While standard reports have their place, there’s a ton to be learned from drilling into your data. Unsurprisingly, we found that sophisticated users of data are more likely to use more filters and breakdowns in their analysis. In fact, the 90th percentile of either Data-Curious or Data-Informed users apply 4+ filters as opposed to a median user that applies 1.9 filters, and they also apply 30% more breakdowns (compared to the average user who only applies 1 breakdown and 2 filters to their reports).

Data-Informed users segment their metrics in 5 different ways (adding filters, editing filters, adding breakdowns, etc.), whereas the median Data-Curious user segments their metrics in only 3 different ways. This suggests that they know what they’re looking for and are laser-focused on a specific set of metrics.

Can’t imagine using just 2 filters?

Same. But this number may hide critical complexity. For example, a single cohort (group of users defined by a chosen set of criteria) may include a number of other filters such as city, operating system, product usage events, and so on. 

“Averages lie, segments (or cohorts) don’t.” 

Ruben Ugarte Practico Analytics

Industry differences

Looking at industry, we found that the number of filters people use during analysis is directly related to how interactive their products are.

That makes sense: While Retail and Consumer Services are selling physical goods or services, for Gaming and Media & Entertainment, their app is the product and therefore requires serious analysis and optimization. As Shreyas Doshi says, “for some companies, the product is the business. For others, it delivers business results.”

Segment differences

Tech Giant users apply the most filters (an average of 2.5), which could indicate their products are dialed in at a high level and they’re looking to make iterative updates based on micro trends in user behavior. In contrast, Scaleups, Digital Transformers, and Startups do simpler segmentation, which can be a powerful approach for accounts with less data.

The takeaway

Data-Informed users take a sophisticated approach to analysis, using filters and breakdowns to leverage every bit—or byte—of data they can. While many answers can be discovered through simple analysis, segmenting your data may be the key that unlocks game-changing insights. 

More important, though, is understanding when the questions you need to answer require a holistic view of the data (in which case, use breakdowns) and when you’re only interested in a subset of your users, or data (in which case, filters will be your friend).

Depth of analysis

How can I push my product analytics skills further?

If you want to improve using product analytics, things like spending more time with your data, trying non-standard data visualizations, and segmenting your data further are no-brainers. However, data-savvy teams also rely on the growth of their most important user cohorts as a measurement of product success. Our Data-Informed users rely on growth KPIs at a rate of 3x more than our Data-Curious users.

What is a growth KPI?

Growth KPIs are multi-dimensional, time- and activity-based metrics. For example, an e-commerce company might track a cohort of users who placed at least 3 orders in a rolling 30-day period, or a SaaS company might track cohorts of users who sent 6 contracts and used 2+ features within 1 week.

By using custom events and custom properties, users can create metrics that weren’t possible before for themselves and the rest of the company to use. We found that Data-Informed users create more than 10 custom events and custom properties, nearly 2x more than Data-Curious users. Once they’re created, custom events are used in 1 in 5 of all report queries, and are used marginally more by Data-Curious users. That suggests Data-Informed users may prefer to work with raw data, but set their whole team up for success by ensuring that everyone doing analysis has the data sets they need.

What are custom events and custom properties?

Custom events and custom properties let you combine existing events or properties into new events or properties on the fly. You can then use these new events or properties almost anywhere that you can use regular events or properties, with the ability to save/share them for reuse across your team.

Segment differences

Users at Startups are the most likely to rely on the growth of their key user cohorts, while Digital Transformers are the least dependent, likely due to higher fluctuation rates. Also, users at Tech Giants create the most custom events and properties, while Startups create the least, which may be because Tech Giants’ teams are dedicated to advanced analytics, while Startups have to prioritize headcount, or it could be that Startups have fewer events and properties that are tracked in the first place. 

That said, Mixpanel’s advanced capabilities can reduce reliance on analysts since many questions can be answered without SQL, allowing Startups to do more advanced queries without a dedicated analytics team, or Tech Giants to move more quickly with distributed self-serve analytics.

The takeaway

Become a power user of product analytics by pushing the envelope on your analysis with things like growth KPIs, creating custom events, and custom properties. These features are incredibly powerful because they enable you to capture important details that are unique to your product. While Data-Informed users tend to take the lead in setting them up, any product analytics user can be successful with them.

Depth of analysis

Should my team use cohorts?

If you’re monitoring how various user groups navigate through your product, then leveraging cohorts is table stakes. The median Data-Informed user saves 4 cohorts and updates existing cohorts more than 200 times per year—7x more often than Data-Curious users. They also use them in reports 4x more frequently than Data-Curious users. Data-Informed users are more interested in closely tracking user segments than Data-Curious users are, likely because they want to identify specific behaviors they can nudge other users to replicate, or help steer users away from behaviors that lead to disengagement.

“Fundamentally, the value of product analysis comes down to understanding customers. We have to get close to the customers and understand what they’re doing, and which of those actions are most important to the direction your company is going. It’s about more than just saying, “what are the most obvious things on the app that we can record”—those things aren’t necessarily indicative of what’s important to your customer.”

James Mayes Co-Founder & CEO at Mind the Product

Explore Mixpanel

See how a Finance company uses cohorts to define power users and churn risks, and plot how they change over time.

Try cohort analysis with sample data

Segment differences

Users at Startups are 25% more likely to update cohorts than Tech Giants, who do this the least frequently, suggesting that Startups are interested in micro-level user trends that can help them better understand, cater to, and encourage valuable user behaviors, as well as find lookalike audiences. Tech Giants still use cohorts very frequently, but experiment with new cohort definitions less often, as they likely already know who their core users are.

“Segmentation is the single most impactful thing product teams can do to improve their onboarding experience. And the only way to know that changes in the user onboarding are working is through cohort analysis. It allows you to see how actions users in the cohort took or take translate into changes into activation and retention metrics. Simply put, if you’re not leveraging cohorts in your analysis, you’re most likely just ‘spraying-and-praying’ in improving your onboarding.”

Ramli John Managing Director at ProductLed Institute

“For startups, it’s likely that your user set, while smaller, is actually more diverse than it one day might be. Using cohorts does two great things: One, it allows you to speak in a more targeted way to your customers; some of which might be there for different reasons; Two, it allows you to learn about the various cohorts and ultimately decide which one (or ones) to focus on as you scale. Most great software companies have a well-defined, almost narrow definition of their ICP. The more testing and experimentation you can do early on, the faster you will identify who your best customers are. Cohorts are a great way to segment the data and learn quickly.

Egan Montgomery Director of Go-to-Market at High Alpha

The takeaway

As data dependency grows, so does the reliance on cohorts. To become a more advanced user of product analytics, compare your data through the lens of different groups of users. Cohorts provide important insights into who’s using your products and which user behaviors are related to success. That’s critical information for any team, as it can guide programs from product development, to onboarding, and customer education.

Resource from Mind the Product

7 Ways Cohort Analysis can Optimize Company Performance and Results

Dan Schoenbaum and Evan Kaeding explain the importance of data and how cohort analysis can improve brand performance.