How to iterate on a product

The most exciting product announcements in a company are usually related to big, eye-catching new feature releases. But another, equally important type of product update tends to fly under the radar: incremental improvements to product functionality.

And even though product builders may see the value of both of these types of new releases, they can still be unsure which one to focus on. How do you—and when should you—prioritize one over the other? How can you show the value of incrementally improving functionality over time, as compared to launching net-new features?

At Mixpanel, we call these incremental improvements our Building Block Updates. Below, we’ll explore a few practical ways to ensure you’re building for long-term success and also share our approach to incremental iteration.

First, let’s look at how to prioritize different types of product updates. Yes, you should smooth out rough edges when you identify them—but you can’t just be smoothing out edges. Here are a few considerations for when you’re facing what we call the power-usability tradeoff, or how to prioritize iterative improvements against flashier feature launches:

Feedback from customers: If a certain feature is causing users to complain (via CSAT surveys or direct feedback to your support team), that’s a good sign that you should prioritize this improvement over a big-swing feature launch.

The data: Examine data like usage behavior analytics to identify areas where users may be dropping off or experiencing friction—and more importantly, whether these issues are impacting business-critical metrics like conversion rates. If they’re not, then you may not have to prioritize this incremental improvement so urgently (especially if you have another big feature launch in the same sprint).

Resource constraints: If we could, product teams would solve every challenge for our users, instantly. But we’re often limited in our resources, whether that’s a lack of time or not enough developers. If you only have enough resources to achieve a quick win, then it might make sense to just go ahead with that—even if it wouldn’t be your priority otherwise.

Sometimes, even if an incremental improvement in the product can’t be tied directly to a key metric, we will still push it through because we genuinely believe it’ll positively impact the user experience in a significant way. But every instance is unique, and the challenge for product teams is finding that balance; there’s rarely a black-or-white answer.

As a rule of thumb, when you’re weighing a big feature release against a few incremental improvement updates, we say that if you can’t easily make a strong case (ideally backed by data) for how the small updates will materially help the business, then prioritize the big release—you can always come back to those incremental improvements in the next sprint after you ship the big one first.

It can be hard to make those arguments for focusing on small incremental changes to functionality. Questions like, “If we adjust the styling of our buttons, would it improve the user experience to the extent that our NPS increases?” may seem too small in ambition or scope. But these incremental improvements add up and, we would argue, are worth designing a process around.

Here are a few steps that’ll help you identify and make small improvements that might otherwise have flown under your team’s radar:

Before you begin iterating, make sure you’ve created a flexible enough process—on both the engineering and design side—that can handle all the potential complexities. Sometimes, small updates can turn into big lifts, especially when you’re dealing with concurrent product updates on your roadmap.

We try to embrace an analytics-driven practice in our daily work. (No surprise, given we’re an analytics company!) That means we focus on building a system that provides democratic access to product data insights. With analysis available to both engineering teams and design teams, everyone stays informed during iteration, which helps with responsiveness and flexibility.

They’re key collaborators, and you should have a clear sense of how you’ll allocate engineers to work on incremental improvement projects while also giving them space for other feature requests. How can you set aside dev resources effectively, negotiate with other internal teams, and convince your engineering team to staff, say, four engineers on incremental improvements? How long would you need those engineers staffed there—a month, half a year, or more? These are just a few examples of questions to ask as you’re planning your iterative projects over the long term. 

Next, identify your most important user flows in the product, which will then lead you toward the metrics you should track. For example, the onboarding flow is important to us because successful onboarding is a key indicator of increased user retention and lifetime value. For this part of the product, we might decide to tie in metrics like engagement rate or task success rate, which we can track through Mixpanel reports:

From here, dig further into different parts of that user journey—specifically, the tasks and stages in the workflows from Step 3. How can you improve those processes for users? This will guide your iteration strategy and give you ideas for where you can experiment. 

Remember to track metrics before and after those experiments so that you can get real-time measurements on whether your efforts are paying off. Here, we might check out data like the time-to-complete metric in a Mixpanel report. If the time to complete a task drops after we’ve shipped an iterative improvement feature, then that’s a good sign that it was successful at reducing friction for the user:

There are so many sources of valuable feedback you can use to improve your product. NPS (Net Promoter Score) is a big one for us, but you can also use interviews and other analytics tools to gather qualitative and quantitative data. 

Combining different sources of user feedback helps avoid a common problem in judging the success of incremental improvements: over-reliance on isolated metrics. You could have an amazing number, but if it’s measured in a vacuum, it may not mean much. An extreme example: Say you built a button that automatically generates a random report. You’d have an amazing “time required to create a report” number, but does that give your users what they really need?

By following these five steps over time, you’ll uncover gaps in your user experience that are flying under your team’s radar but may be pain points for your customers.

In Mixpanel, we have different ways to view charts and queries—the single report view and the Board view, which shows several reports together in a dashboard. The problem: Because we built those features in isolation from each other, the Board didn’t support certain types of visualizations. That meant if you clicked a certain chart like a bar chart on your dashboard, you might land on a page showing a line graph that looked completely different.

Our users didn’t like that. It was confusing, and they (understandably) wanted to see consistent visualizations of their data no matter where in the product they were. They weren’t the only ones impacted either—this also created more work for our engineers because they had to maintain more disparate systems, and our design team ended up designing more charts and visuals than they may have needed to.

We solved the problem for users and engineers by revamping our Boards feature. In the grand scheme of things, this was more of an iterative improvement than a big headliner feature release. But it helped multiple stakeholders in a significant way and is a great example of when you should try to prioritize these smaller updates.

Even though you may not always see this type of work highlighted in feature requests, it can often still be impactful down the line when users notice, for example, a much smoother login or signup experience. And equally as important, it can give you a significant product advantage over competitors. These are benefits that directly impact the business—and if we looked only at conversion or engagement rates, we’d miss out on this feedback.

Product and engineering teams come in all shapes and sizes, so spending a fair amount of time on Step 1 and Step 2 above becomes important for finding the right incremental product improvement frequency for yours. No matter what cadence you can manage, maintaining a holistic approach that incorporates user feedback, data-driven decision-making, and a strategic allocation of resources will make your iterative dev efforts the most effective. This way, it’s possible to balance ongoing, impactful improvements with net-new features and, ultimately, create a more resilient and user-friendly product.