Choosing, refining, and tracking product metrics

Why do smart product teams often build products with mediocre or no impact?  Often, the answer is found in the discipline with which a product team measures the product. 

When selecting your metrics, you’ll want to consider a few different categories. These categories cover multiple granularities and perspectives that will help you make rigorous product decisions. In general, the following categories address questions that will give you a well-rounded understanding of the product’s value for its users and your business.

Note, a given metric can belong to more than one category. At the same time, it may make sense to skip a certain category altogether for your product, depending on the product’s unique context and its maturity.

It’s vital that your team understands how well the product is performing, and is aware of potential issues. 

Key question addressed: Is the product available and performing in the manner that users would reasonably expect?

Common examples that fall within this category include:

The success of a product largely depends on how users engage with it, as well as how often (and if) they come back for more. Usage metrics should almost always be among the most vital inputs for product decisions. 

Key question addressed: How are users using the product?

Examples include:

Adoption metrics give you a view into the user experience, but from a different perspective than usage metrics. They focus on how your users adopt the product and how they react to a new feature or service.

Key question addressed: Is the product (along with its key features) being used as much as we’d hope, and in the ways that we’d like?

Teams often track the following adoption metrics:

Satisfied customers are most likely to spread the word about your product. Customer satisfaction impacts not only your other key metrics but also your company’s brand.

Key question addressed: What is our customers’ overall sentiment towards the product or its main features?

Examples include:

A product doesn’t operate in isolation. Tracking a product’s position within its macro-environment can help teams keep their eyes on bigger opportunities and long-term planning.

Key question addressed: What is the macro state of the product within the domain in which it operates?

Common metrics to regularly track are:

"For some companies, the product is the business. For others, it delivers business results."  

Key question addressed: What overall business results are we seeing from this product?

The metrics chosen are unique to each business, but often include:

For any product, you’ll likely track dozens of metrics (especially in the usage metrics category). At the same time, you don’t want to be inundated with data to the point where it's difficult to see the big picture.

I recommend defining a small set of metrics (3-5) that'll be faithful indicators of your product’s progress towards its strategy and goals. These are your Key Metrics (KMs).

At the highest level, a company typically chooses a north star metric that’s responsive to product changes, and is a holistic representation of the product’s value for users.

However, a north star doesn’t live alone. Rather, metrics measure a complex product with different dimensions. As Reforge CEO Brian Balfour says, “most products should have four high-level metrics: an acquisition metric, retention metric, engagement metric, and monetization metric. These four act as a system where one can influence the other.”

By capturing a different dimension within each product, product managers can better understand the context behind the numbers, and find a balance that will move the product forward. 

When it comes to the criteria for choosing your key metrics, there isn’t a set of hard rules. The criteria I use for key metrics are:

KMs give a great bird’s-eye view of how the product is doing. You’ll typically create annual goals and targets for your KMs. Your company’s executives and board will also likely look at these metrics to assess the product’s progress. 

Some teams avoid using a certain indicator as a KM because there are outside factors that impact the metric. For example, a team will say: “for this, I depend on sales, so I can't use it as a KM." It’s true that sales impacts some KMs you might choose, but I don’t worry about having 100% control. There's no such thing as 100% control anyway. 

"In healthy companies with a strong sense of individual ownership, the thinking goes: If sales can’t or doesn’t sell my product, that’s still my problem."

In certain cases (e.g. B2B SaaS with long adoption cycles), KMs are lagging indicators of work done (e.g. revenue generation). With long sales cycles, it’s hard to confirm how features or performance is impacting revenue. 

That’s why teams also need a small set of leading indicators, which I call Leading Metrics (LMs). 

LMs help you understand activity, changes, and experiments within a shorter time frame. 

For example, over the next two to three quarters, you might have an LM for adoption of a particular strategic feature (which over time will move your KMs), an LM for country availability, or an LM for self-serve funnel success.

For companies that choose KMs that are lagging indicators, I recommend also defining 3-5 LMs at a time. LMs can change more frequently as they measure overall product progress and business growth. 

Now that you’ve chosen and refined your metrics, it’s time to put them to use. Some tactics I’ve found useful for using metrics:

When setting metrics, beware of false precision. Often, when it comes time to project our metrics for next year, we put in a lot of work to pretend that we’re being highly analytical. The reality is that there’s no way to accurately predict certain numbers. Too much energy spent on precise predictions is a waste for most early- and mid-stage products. 

When in doubt, set qualitative goals that are the closest possible proxy to the numbers you want to achieve.

To sum up my key advice for those leading their team in choosing the right product metrics:

The ability to break the rules—and to know when—is often what distinguishes great PMs from the others. A great PM will diligently track metrics, but will rely on them as one source of input among many.