Using data for successful decision making

When presented with dozens of options to serve the needs of users and stakeholders, product managers are uniquely empowered to make decisions.

And as data scientist Cassie Kozyrkov puts it, “a ‘decision-maker’ is not that stakeholder or investor who swoops in to veto the machinations of the project team, but rather the person who is responsible for decision architecture and context framing. In other words, a creator of meticulously-phrased objectives as opposed to their destroyer.”

And to be a successful decision maker who creates rather than destroys, product managers need to responsibly gather, interpret, and weigh the value of quantitative and qualitative user data—often employing both methods to determine what new features should be introduced to who and when. 

Here’s how you can use both data collection methods to help you on your path to making informed decisions.

Step 1: Validate your thesis

No one should be building products based on a hunch. And qualitative data is one of the best ways to go about validating your thesis about what users need and why. 

Talking to customers will test assumptions about what you think is valuable, especially if you’re asking questions that avoid the “counterfeit yes.”

For example, product manager Clement Kao found that the need for more metrics wasn’t going to improve user churn for a CRM built to help real estate agents close deals. “We had plenty of metrics,” Kai explained in his post detailing user analytics. 

It wasn’t until Kao started shadowing agents that he “noticed that the most engaged users interacted with the app very differently than the disengaged cohort.” That qualitative analysis gave him the insight to measure how engaged and disengaged cohorts differed. 

Step 2: Scope the problem

Only quantitative data can show the scope of a problem. 

In Kao’s case, the quantitative data confirmed his suspicions from his field research: an engaged user behaves much differently than a disengaged user. The results also tested a lot of core assumptions the company made about its users. 

“The data proved these assumptions completely wrong,” Kao said. “In reality, our most engaged agents rarely, if ever, used our prioritization feature to order their pipeline by priority. Instead, they frequently rejected leads. They also often reassigned leads to other team members. Finally, highly engaged users would regularly set vacations in the app.”

By measuring user analytics with Mixpanel, product teams can also gain similar quantitative insight from their users.

Step 3: Create a story

To sell a new feature, you need to be able to tell a great story. And qualitative research helps those who may be disconnected from customers day-to-day (such as engineers and managers) build empathy for users. It’s data that helps teams understand how users feel and why they feel the way they do.

Meanwhile, hard data from quantitative research keeps the story you’re telling rooted in reality. It concretely shows what conflicts exist (such as number of users not completing a certain task) and what business issues are immediately solved.

A well-constructed story also shows stakeholders why you’re the best person to advocate for and eventually deliver a feature that drives significant impact for users. 

By leveraging quantitative and qualitative data, you have created a foundation built on facts and experiences, setting yourself up to be a meticulous creator who grows products. And to accurately gauge if your product decisions have actual impact on users, Mixpanel’s Impact report will give you the quantitative data you need to move forward.