Should I segment my product data?
While standard reports have their place, there’s a ton to be learned from drilling into your data. Unsurprisingly, we found that sophisticated users of data are more likely to use more filters and breakdowns in their analysis. In fact, the 90th percentile of either Data-Curious or Data-Informed users apply 4+ filters as opposed to a median user that applies 1.9 filters, and they also apply 30% more breakdowns (compared to the average user who only applies 1 breakdown and 2 filters to their reports).
Data-Informed users segment their metrics in 5 different ways (adding filters, editing filters, adding breakdowns, etc.), whereas the median Data-Curious user segments their metrics in only 3 different ways. This suggests that they know what they’re looking for and are laser-focused on a specific set of metrics.
Can’t imagine using just 2 filters?
Same. But this number may hide critical complexity. For example, a single cohort (group of users defined by a chosen set of criteria) may include a number of other filters such as city, operating system, product usage events, and so on.
Looking at industry, we found that the number of filters people use during analysis is directly related to how interactive their products are.
That makes sense: While Retail and Consumer Services are selling physical goods or services, for Gaming and Media & Entertainment, their app is the product and therefore requires serious analysis and optimization. As Shreyas Doshi says, “for some companies, the product is the business. For others, it delivers business results.”
Tech Giant users apply the most filters (an average of 2.5), which could indicate their products are dialed in at a high level and they’re looking to make iterative updates based on micro trends in user behavior. In contrast, Scaleups, Digital Transformers, and Startups do simpler segmentation, which can be a powerful approach for accounts with less data.
Data-Informed users take a sophisticated approach to analysis, using filters and breakdowns to leverage every bit—or byte—of data they can. While many answers can be discovered through simple analysis, segmenting your data may be the key that unlocks game-changing insights.
More important, though, is understanding when the questions you need to answer require a holistic view of the data (in which case, use breakdowns) and when you’re only interested in a subset of your users, or data (in which case, filters will be your friend).
How can I push my product analytics skills further?
If you want to improve using product analytics, things like spending more time with your data, trying non-standard data visualizations, and segmenting your data further are no-brainers. However, data-savvy teams also rely on the growth of their most important user cohorts as a measurement of product success. Our Data-Informed users rely on growth KPIs at a rate of 3x more than our Data-Curious users.
What is a growth KPI?
Growth KPIs are multi-dimensional, time- and activity-based metrics. For example, an e-commerce company might track a cohort of users who placed at least 3 orders in a rolling 30-day period, or a SaaS company might track cohorts of users who sent 6 contracts and used 2+ features within 1 week.
By using custom events and custom properties, users can create metrics that weren’t possible before for themselves and the rest of the company to use. We found that Data-Informed users create more than 10 custom events and custom properties, nearly 2x more than Data-Curious users. Once they’re created, custom events are used in 1 in 5 of all report queries, and are used marginally more by Data-Curious users. That suggests Data-Informed users may prefer to work with raw data, but set their whole team up for success by ensuring that everyone doing analysis has the data sets they need.
What are custom events and custom properties?
Custom events and custom properties let you combine existing events or properties into new events or properties on the fly. You can then use these new events or properties almost anywhere that you can use regular events or properties, with the ability to save/share them for reuse across your team.
Users at Startups are the most likely to rely on the growth of their key user cohorts, while Digital Transformers are the least dependent, likely due to higher fluctuation rates. Also, users at Tech Giants create the most custom events and properties, while Startups create the least, which may be because Tech Giants’ teams are dedicated to advanced analytics, while Startups have to prioritize headcount, or it could be that Startups have fewer events and properties that are tracked in the first place.
That said, Mixpanel’s advanced capabilities can reduce reliance on analysts since many questions can be answered without SQL, allowing Startups to do more advanced queries without a dedicated analytics team, or Tech Giants to move more quickly with distributed self-serve analytics.
Become a power user of product analytics by pushing the envelope on your analysis with things like growth KPIs, creating custom events, and custom properties. These features are incredibly powerful because they enable you to capture important details that are unique to your product. While Data-Informed users tend to take the lead in setting them up, any product analytics user can be successful with them.
Should my team use cohorts?
If you’re monitoring how various user groups navigate through your product, then leveraging cohorts is table stakes. The median Data-Informed user saves 4 cohorts and updates existing cohorts more than 200 times per year—7x more often than Data-Curious users. They also use them in reports 4x more frequently than Data-Curious users. Data-Informed users are more interested in closely tracking user segments than Data-Curious users are, likely because they want to identify specific behaviors they can nudge other users to replicate, or help steer users away from behaviors that lead to disengagement.
See how a Finance company uses cohorts to define power users and churn risks, and plot how they change over time.Try cohort analysis with sample data
Users at Startups are 25% more likely to update cohorts than Tech Giants, who do this the least frequently, suggesting that Startups are interested in micro-level user trends that can help them better understand, cater to, and encourage valuable user behaviors, as well as find lookalike audiences. Tech Giants still use cohorts very frequently, but experiment with new cohort definitions less often, as they likely already know who their core users are.
As data dependency grows, so does the reliance on cohorts. To become a more advanced user of product analytics, compare your data through the lens of different groups of users. Cohorts provide important insights into who’s using your products and which user behaviors are related to success. That’s critical information for any team, as it can guide programs from product development, to onboarding, and customer education.
Resource from Mind the Product
Dan Schoenbaum and Evan Kaeding explain the importance of data and how cohort analysis can improve brand performance.