Tips when asking for product feedback

Have you ever told someone what they wanted to hear because it was the path of least resistance? 

“Yes, let’s totally get lunch next week! I’ll text you.” (And then you ghost.)

“Yep, sounds like a good idea.” (But you don’t fully understand it.)

It’s always easier to say “yes,” especially when you don’t have to follow through right away. It’s an exit from the conversation that makes both people feel good.

But pleasant talk turns “yes” into a dangerous word if you’re collecting product feedback. If you think about it, every feedback conversation is a negotiation in disguise. You want honest information from a customer, but they may be uncomfortable disclosing exactly how they feel, for a variety of reasons. Often, they know you’ve worked really hard on the idea you’re running by them and don’t want to disappoint you. But what you want isn’t an easy or kind response. You’re after nuanced and honest.

In these types of situations, consider what the FBI’s former chief hostage negotiator, Chris Voss, calls “the counterfeit yes.”

“A counterfeit ‘yes’ is one in which your counterpart plans on saying ‘no’ but feels ‘yes’ is an easier escape route.”

In Voss’ line of life-or-death work, knowing the truth about what the other side really wants and means is the only way to reach a peaceful resolution.

While the stakes obviously aren’t as high in digital product development, the path to finding a problem-solution fit is no different. Having rolled out plenty of feature launches here at Mixpanel, we've seen a counterfeit yes or two (or ten) and have fallen for them ourselves. You’ll know you’ve made a mistake when you try to drive adoption of the well-intentioned feature but come up short because the idea wasn’t thoroughly vetted.

You need to ask the right questions, because false information, no matter how kindly intended, leads to features that miss the mark. 

Here’s an example:

At first glance, you might think that a question posed in such a way would lead to a pretty clear and useful customer decision: yes or no. But consider this: When posed with the possibility of a new feature that might help them, customers usually default to saying “yes.” If it’s not useful, they won’t use it, so what’s the harm? 

Here’s the harm: If you build one thing, it means you aren’t building one of the other five things on your list – at least not right away. Getting genuine feedback on your ideas is essential because you need to prioritize accurately.

Tip: Avoid asking leading questions with easy answers (read on for some ideas…).

This is a more open-ended question that eliminates the easy out. The customer will need to explain the value they expect to receive, and it will be abundantly clear if there isn’t a use case match. 

Tip: Apply the Rule of Three and ask the same thing in different ways. It’s one of Voss’ techniques for discerning whether the other party is telling the truth, or taking an easy out.

“The Rule of Three is simply getting the other guy to agree to the same thing three times in the same conversation. It’s tripling the strength of whatever dynamic you’re trying to drill into at the moment. In doing so, it uncovers problems before they happen. It’s really hard to repeatedly lie or fake conviction.”

When you pose the question this way, you get to the heart of whether the idea is actually valuable to a customer. Not every feature comes with additional cost, but if it will, you should definitely ask. Think about sitting down at a new Italian restaurant in town, where the waiter promptly asks if you’d like bread. Would your answer change if it wasn’t free, but $5 instead?

Tip: When you need an honest answer to a critical question, try to do it in person or over a video conference to get the best read on their response. A famous UCLA study led to the 7-38-55 rule for communication, which represents that “7 percent of a message is based on the words while 38 percent comes from the tone of voice and 55 percent from the speaker’s body language and face.”

We once spoke to someone who worked at a video conferencing company where many of their customers also used Salesforce. So they asked users, “Would you like the ability to launch a video conference from Salesforce?” Everyone they talked to said, “yes,” so the product was viewed as a no-brainer and built quickly. 

At launch they decided to charge a percentage of the existing contract value for the new, add-on product, and only a few companies bought it. When they rethought the strategy and made it free to customers spending over a certain amount annually, the feature still continued collecting dust.

It turned out that the feature simply wasn’t valuable enough to invest time in the setup process. It typically involved hassling the Salesforce admin to enable the integration, and these people were always flooded with requests. Calling in a favor usually wasn’t worth it, so very few people acted.

Ultimately, the product was retired and the company moved on to bigger and better things. But all that time could have been spent elsewhere if the counterfeit yes’s had been discovered through better questioning upfront.

So now that you know how to avoid the counterfeit yes, what do you do with answers you get?

In his 25 years leading high-stakes negotiations, Chris Voss came to view a ”no” answer as the most useful starting place for getting accurate information.

The word has a liberating quality that makes the person being questioned feel in control—and a sense of control leads to more comfort and more honesty. 

As a PM who wants to make decisions based on facts, nothing should be more comforting than hearing “no”—even if it’s in response to a feature idea you worked hard to come up with. While it may sting at first, you saved yourself and your team loads of time, and in the process, developed a trusting relationship with a customer.

Tip: Early on in a feedback conversation, deploy a “no”-inducing question that puts the other party in control (e.g., “Do you feel you wouldn’t be able to do your job as well without this feature?”). If they say “no,” you can ask “Why not?”—an effective launch pad for gathering the information you need.

You can’t talk to everybody, and sometimes reaching a statistically significant sample of conversations would even be impossible. Collecting qualitative feedback at scale via surveys is often a good idea, but you can’t bombard your user base with a questionnaire for everything.

Furthermore, the saying that “actions speak louder than words” exists for a reason. It may be a bit extreme here, but the point remains that merely asking people how they’d act can lead you astray.

Product usage data can complement the qualitative data you collect and provide quick insights on users’ pain points and opportunity areas. Products like Mixpanel help you look at user behavior data in great detail to learn with certainty how people spend their time in your product, and which features actually invoke deeper engagement.

Tip: When you give users access to a feature in Alpha or Beta, check the usage patterns. Are people using it as you intended? Are they coming back to use it again and again? If the early indicators look good, roll it out further and keep monitoring the data. If not, reach out to users and have a few conversations to learn more. 

As we’ve covered, there are many reasons why well-meaning customers may end up misleading you. Perhaps they aren’t sure if they’d want your feature, maybe they aren’t considering the tradeoffs required to pursue it, or maybe they don’t want to disappoint you. Regardless of the reason, your success depends on uncovering the unvarnished truth. On your path to the facts, don’t forget to ask the right questions, and beware of the “counterfeit yes.”