How to establish a product analytics practice
By embracing product analytics as a practice, teams can make self-serve data the cornerstone of their decision-making and turn the sea of unknowns into knowns.
Product analytics promises us something incredible: With more behavioral data about your users, you can make more informed product decisions.
But what if you feel like you’re drowning in data? If you have a mature, well-established product, it’s even harder to mine through millions of end users and billions of data points. Or what if you have no data but need to generate insights quickly? Regardless of your data maturity, it can be hard to know where to start.
By embracing product analytics as a practice, you can turn the sea of unknowns into knowns. A product analytics practice is a culture of making self-serve data the cornerstone of your decision-making. You focus on access, standardization, and turning your findings into action. In other words, “Let the data be your guide.”
Establishing a product analytics practice is a process. It starts with changing your internal approach to data. The result? A better understanding of what the data is telling you and more clarity as you make product decisions.
Why establish a product analytics practice?
At Mixpanel, we hear a lot of the same complaints when we work with product teams and other cross-functional teams within a company:
- They don’t have access to the data they need.
- They’re stuck waiting for answers from a centralized data science or analyst team.
- Even if they have access to data, analysis requires data context and/or SQL knowledge.
- Product experiments and feature launches take too long to measure success.
Some of these companies have even implemented product analytics but don’t have strong internal practices. In that case, it’s just another tool in the tech stack and doesn’t drive value. They don’t see any real impacts on their product development.
As statistician and economist E.F. Schumacher said, “An ounce of practice is generally worth more than a ton of theory.” Approaching product analytics with a plan can be a game-changer. After all, the goal of product analytics is to make your team’s life easier. But that change doesn’t happen on its own—it needs effort and resources from your team.
4 steps to establish a product analytics practice
Product analytics is not a one-time implementation project. Instead, it needs to become an integral part of your workflow, processes, and culture. A good analytics practice needs everyone on your team to change their approach to data.
It’s not as complicated as you might think. You can make small tweaks to your interactions with data and gradually introduce bigger shifts. Before you know it, you’ll be able to place large-scale product bets—all based on data.
“Our teams are able to make thousands of good decisions because people have access to data in an easy and efficient way.”
Nikita Strezhnev, Data Analytics Manager @ Bolt Food & Delivery
Whether you’ve yet to establish a practice around product analytics or your current processes aren’t working, here are four steps you can take to improve your use of product analytics.
1. Start with a simple use case
Getting value from product analytics never starts with big, sweeping changes. It starts with practical questions. Take a simple use case and help your team understand end-to-end how they can use product analytics to answer a key question about your users. The complex questions can come later.
Your flow should be:
- Ask a question
- Implement data to answer
- Build a report
- Generate insights
- Use case solved
- Repeat
At Mixpanel, too often we’ve worked with customers that insist on adding every interaction (events) users can take on their platform. The logic is that the interactions “might be important.” But this results in implementations taking far too long to complete and review. Months go by before users even get to start answering questions. And most of these events never end up being queried.
As a result, the product analytics tool ends up being bloated with events that confuse users. Teams get frustrated because they can’t easily find events that they care about. Meanwhile, it’s taking longer than it should to see ROI.
When companies start small, users can quickly reduce time-to-value because they can answer key questions. Then they iterate by adding additional events to answer deeper questions or analyze more use cases.
Your team should look at events and ask themselves, “Why?”. For example, by tracking “signup” and “add to cart,” you can answer questions like “How many daily site visitors add items to the cart?” or “How long does it take for a user to convert from signup to add to cart?”
With a few events, they can measure the following metrics:
- The number of users who purchase at least three times
- Users moving through different lifecycle stages, such as new, engaged, dormant, etc.
- Purchase conversion rate or time to convert
- Retention rate of users
Before product analytics, you may have been making assumptions about how users get from point A to point B. That might be anything from a subscription signup to using a specific feature.
Each step the user takes in your product is a unique event—and it’s shocking how quickly events can add up. That’s why starting with a simple use case is the best way to understand product analytics in the beginning. You’re narrowing your criteria to a series of one to three events.
Did the users complete the steps you’ve always expected? Or were there some surprises?
This is the first goal: to find a simple business case that you can improve (such as reducing the number of events before purchase, or nudging users in a specific direction). If you have multiple teams that will be using product analytics, start with one team and make them successful for their key metrics first.
Product analytics depends on your team’s ability to explore. They need to understand the story being told through data before they can apply it to product recommendations.
2. Implement data governance best practices
It’s hard to work with messy data. And it’s equally hard to go back and fix messy data. If you can implement data governance best practices in the beginning, your product analytics tool will be far easier to work with (and give you better insights). Otherwise, you risk looking at an incomplete or inaccurate picture.
Even if you implement product analytics incrementally, it’s important to establish standards that will apply across the board. There are likely some core datasets that will be relied upon by multiple teams, so it’s critical that everyone can use the data effectively.
Some of the details include:
Data source: Determining how the data will flow from your source of truth into your product analytics tool (such as data warehouses, transactional systems, or server-side logs)
Consistent naming conventions: Giving your events and properties easy-to-understand and organizationally accepted names to make navigation and reporting easier
Data transformation: Think about fields that may need standardization (such as a date field) across multiple datasets
Distinct IDs: Identifying the unique identifiers for users or any global values that are included in your datasets
Security: Outlining any requirements, including any personal identifiable information that may need to be protected or follow specific data retention requirements
Governance owner: Identify a data governance owner who will run regular data health checks
Environment: Keeping your production data separate from your development data
If you’re working with data that hasn’t had some best practices implemented, take a similar approach to Step 1: Start small. See if there’s a simple way that some data can be cleaned up to build a simple use case. This will allow you to better understand the issues with the current data and build a plan towards stronger data governance.
By starting small, your team can really understand the data and prove that your approach to analytics is working before moving on to more use cases and teams.
3. Democratize product analyses
Compiling the data is one thing; using it is another.
You’ll want to democratize the data, or make it accessible and understandable for your team. If the underlying data structure is complex, it makes it hard for your team to work with the data. When you rely on a data scientist or analyst to wrangle some queries to produce insights, you lack data democracy.
You also risk that something is lost in translation between the person who understands the use case and the person working with the data. A data scientist or analyst may not have the same depth of knowledge about the topic as the team that submitted the question. With data democracy, that problem is removed.
Ideally, you’ll appoint an internal company champion to oversee the adoption of product analytics. This would include setting up onboarding for new users, training users on data governance best practices, and creating resources like videos and dashboards. Your champion becomes the “go-to” person that can answer the team’s specific questions about the product analytics tool.
“Mixpanel is a gold mine. It is a must-have for organizations developing data-informed cultures. It allows our team to work from one data source, speak the same language, and see clearly what makes sense to build–and especially what doesn’t.”
Paolo Sabatinelli, Chief Product Officer @ Immobiliare
Democratizing data also means breaking down data silos between teams, product lines, and data sources. If a piece of the puzzle is missing, you can’t make an informed business decision. A unified data model brings all of your data to a centralized repository so that everyone can have access.
Too often, product teams are stuck waiting on a data science team that can handle complex queries. Data democratization is the first part of reducing this bottleneck. The second part is self-serve analytics. Think of it this way: Data democracy refers to access, and self-serve analytics refers to usage.
Self-serve analytics refers to the ability to create ad hoc reports, build custom dashboards, and perform data analysis without any support from a data analyst. Stakeholders have the information they need to make informed decisions.
“In just a few clicks, anyone from any part of the organization—product, engineering, operations, local teams, and central management—can self-serve answers they need about our users… There’s no need to wait for an analyst or data scientist to design and run a query; everyone can get fast answers and agree on a way forward.”
Nikita Strezhnev, Data Analytics Manager @ Bolt Food & Delivery
As an organization, you’ll need to invest in data literacy. Teams will need to understand how your data is structured and how events are tied to the user actions within the product. You’ll need to make sure that your teams are trained on self-serve analytics so that they’re comfortable answering their own questions.
The goal of data democracy is to put trusted data in the hands of as many people as possible.
4. Build a culture of experimentation
When the data is self-serve, teams can rely on real-time analytics, or data that is updated within seconds of the event happening in your product.1 Real-time data shows how users interact with the product so you can react to critical issues, such as a product launch or experiment.
Whether you’re trying to respond to issues in real time or simply looking to reduce the length of your feedback cycle, product analytics can help. But your team’s access to data is only as effective as their ability to make decisions.
If there are too many internal barriers, you lose some of the value that product analytics can provide. The time between identifying a solution and pushing a solution out to your users should decrease as your team becomes more comfortable with product analytics. If users aren’t blocked by stretched data teams, they’ll be able to move more quickly.
Identify the ownership for decision-making based on product analytics. This might include incorporating product analytics into your PRDs (Product Requirements Docs), presenting insights to a team for feedback, or iterating quickly on new features as they’re released.
Yelp was using an analytics practice with a centralized analyst team running queries for their product managers, which slowed down its decision-making process. With Mixpanel, Yelp reduced the time to conduct product analysis from weeks to a few days. Yelp was also able to decrease the time between ideation and launch for critical features.
“Mixpanel helps us make better product decisions faster, supercharging the investments we’re making.”
Sid Arora, Head of Data Products – Analytics and Experimentation @ Yelp.
Your product analytics practice will eventually reach a place where you move from reacting to what is known to placing bets on the unknown. You can use feature flags to experiment with ease and reduce risk. And you can have more clarity around the outcomes of your experiments by splitting your analytics into two groups: those that received the experiment and those that don’t. That way, your results aren’t skewed by users that weren’t part of the experiment.
The more you rely on self-serve experimentation and easy access to A/B features, the more you can build a culture of testing. Your experimentation can cover a broad range of use cases from feature management to optimizing your customer acquisition strategy.
Data collection and experimentation with everything you ship will become second nature. And with your data governance best practices, democratized access, self-serve analytics, and agile decision-making, everything will continue to build over time—leading you to a place where you can build faster and experiment with confidence.
Plan an incremental rollout
If you go back to your simple use case, your team can work on building out the initial reporting. It starts with a goal for that use case (like increasing signups) and leveraging the data to establish baseline key metrics. As product changes are made, the team can measure and validate the impact using self-serve analytics. And over time, this will become part of your product development process.
You want to create a solid foundation within a single team before moving to additional teams. That team should be comfortable integrating data collection into its product development process, accessing self-serve analytics, and making decisions before you roll out a product analytics practice to additional teams.
As the team is able to measure its metrics using product analytics, celebrate the milestones. From that point, adopting a product analytics practice within additional teams becomes easier. It becomes a collaborative effort: Teams can share their learnings and best practices.
Scale your product analytics practice
A company-wide adoption of a product analytics practice can only be as effective as the effort from everyone involved. From product managers to engineers, there needs to be a focus on creating a culture centered around data. That means that leadership should reinforce its importance and make sure that teams have the necessary resources.
By integrating product analytics into your product reviews, retrospectives, quarterly plans, rituals, and other key milestones, you’ll emphasize the importance of data and its outcomes. Product analytics will take its place as a “must-have” component of your product strategy.
Mixpanel can ensure you’re set up for success with a strong product analytics practice. We provide free onboarding to any company on an enterprise plan, and we have a Slack Community for all users to come and ask questions. And our transparent pricing enables high-growth companies to scale confidently without any surprises.
Check out our pricing page for more information. You can also sign up for free to try Mixpanel yourself.
Some tools have a delay or rely on batch updating from a data warehouse.1
