How to get started with product analytics

In partnership with Skillshare, Mixpanel product manager and ex-Googler Veronica Pinchin created a video course on getting started with product analytics. Called “Product Analytics Essentials”, the class covers what product analytics is (and isn’t),  how to set goals, create an implementation plan, and then use three key reports to find insights on user behavior. We’ve transcribed two of the most critical components of the class here: how to set business goals and build an analytics implementation plan based on those business goals.

Want to get the full end-to-end picture on product analytics essentials? Watch the full class here (it’s less than 40 minutes long!).

How to set business goals

The reason that it’s important to set business goals with product analytics is because there are millions of things that you can track in your app or website. And if you track every single one of those, it’s going to be very time consuming, super expensive and you’re gonna end up with a lot of useless data and not a lot of insights. By setting business goals up front, you can provide a structure and framework for the things that you’re going to track and make sure that you’re getting valuable insights out of your product analytics.
Some example of product-focused business goals are things like:

• User activation: Where are people dropping off when they’re onboarding onto your product?
• User retention: Which users are likely to come back and which users are likely not to come back?
• User referrals: Which users are the most likely to invite other friends to use your product?
• User engagement: Which features are people using the most or the least?

Beyond product goals, there are probably a lot of other goals that you might have for your business, like goals around marketing, finance or sales. All of these goals are part of a continuum that helps really drive the success of your business.
In my own work, I start by looking at what the business is trying to achieve overall. At Mixpanel, one thing that’s really important to us, for example, is making sure that people are coming back to Mixpanel on a daily basis to get value out of the data that they’re collecting. So as part of this, one of the goals that we have is around “viewing reports”. We want lots of people to come back and view a lot of different reports because, for us, that’s a good indicator that they’re getting value out of their data and our product.

One simple example that we’re gonna be using throughout this class is that of a music app, like a Spotify or a Google Play Music. If you think about that kind of app, the things that you might care about are the number of songs that people are playing, how often they’re coming back or maybe the number of people that are upgrading to a paid plan. Those would be your business goals and you can set up your implementation tracking plan to figure out if you’re achieving those goals.

When you’re thinking about how many business goals you want to track, there really is no right answer. If you’re a small website just starting out, you might only have a couple. There might be one or two or three things that you think are really important to driving impact for your business. As you start to scale, that could blow up to having multiple goals across several teams. And so you could end up with a lot more goals than that in a large product analytics implementation.

At Mixpanel, for example, we do have some overarching goals that are really the North Star for our business. But for me specifically, when I’m product managing the machine learning team, I have a specific set of things that I care about that are very different from what another product manager who’s running the mobile team might care about. So those goals that are specific to the machine learning team are really the things that help guide my day-to-day decisions. Choosing two to three goals initially will help you create your implementation plan.

How to create a tracking implementation plan

An implementation plan is a document that helps you map your business goals to specific events and properties that you want to track in order to answer questions about those goals. Your implementation plan is like a map for your developers to implement a product analytics platform like Mixpanel.

Creating an implementation spec is really important because there are so many things that you could track. If you track everything without knowing up front the types of questions you want to be able to answer, you’re going to end up with a lot of data and not really any idea of what to do with all of it.

You actually don’t need to be technical at all to write an implementation spec. All that you have to do is start with those business goals that you defined in the last step.

For each of those goals, break it down into the key questions that you really want to understand about your product and about user behavior in your product. From there, identify the user flows or patterns of behavior that are going to help you answer that question. Once you have those flow identified, all you do is identify the specific events and any properties that you care about that map to those flows. That’s your implementation spec!

You’ll only create your tracking plan once from the ground up since it really is a living document. As you make changes to your product to alter the flows or the key behaviors that you’re driving through your product or as you launch new features, you’ll  go back to that tracking plan and update it with new events and properties that you care about.

Let’s think through the tracking plan for that hypothetical music app. To start, we find that using a spreadsheet to create your tracking plan works really well: it helps you organize both your business goals and the questions you want to answer, along with the events and properties that you want to track. In this case, the first business goal is acquisition: we want people actually logging in and using our music app.

Next, we’re going to look at how you actually populate the tracking plan spreadsheet. First, you need to figure out the key activities that will indicate if you’re achieving your goals. In the case of our music app and our goal of acquisition, the key activity would be completing the signup process. So, we’re going to need at least one event that shows people completing the signup process. There are also some important steps that happen right before signup that we’re going to want to track as well: the user will have to open the app, actually begin the signup process, and finally, complete the sign-up process. These are the key events to track in that flow.

Why is it important to track every step of the user flow? Because if signups are much higher or much lower or even right where you expect them to be, you will want to know why. At each step in this flow, there might be reasons why people are more or less successful at completing them. If, for example, you find out that people logging in with Twitter have a much lower successful signup rate than those logging in with Facebook, you might investigate the process for signing in with a Twitter account and maybe make some improvements there.

Depending on the goal, there can actually be many paths that allow you to understand if people are achieving that goal. In this music app, there might be a few different ways that someone can upgrade to a paid subscription. You’ll want to implement tracking for every one of those flows if upgrading to paid subscriptions is a goal that you care about overall. However, other business goals such as new user acquisition might depend on the one way to sign up. In that case, there might just be one flow that you track.    

For each event that’s tracked, there are probably multiple properties to track, too. These properties provide details about the event that might give you deeper insight when you’re doing your analysis later. For example, with a login event,we might care about how users are logging in, their account type, follower count, or when they last logged in. You might also consider properties like country or device type.

When you’re building your implication spec, you want to include any event and any property that you think might be meaningful when you’re asking key questions about your business.

That being said, it’s human tendency to want to track absolutely everything. If you do that, you do run the risk, at some point, of having so much data that it becomes hard to extract the pieces of the data that are really meaningful and that do drive impact. The great thing about these plans is it they provide the framework to make sure that the data that you do collect is high-quality and understandable to your team when they’re running product analytics reports on their own.

Once your plan is created, it’s time to work with your developer. Depending on your role, you might have a different level of comfort talking to developers about technical considerations. The great thing about an implementation spec is it really provides a map for you to have a constructive conversation with your developer, even if you’re not very technical. No matter how technical you are, you can use the plan to walk through how you would actually track and record each of these different events.

If you actually break the implementation plan down step-by-step and take it one goal at a time, it actually becomes pretty easy, and it gets easier the more that you do it.

******
Next steps: Using common reports like funnels and retention to understand user behavior. Watch the full class here.