Product managers and stand-up comedians - Mixpanel
Product Foundations

Product managers and stand-up comedians

Jordan Carr

Alpha Founder & CEO Thor Ernstsson’s career in tech started with a move from Iceland to Alabama, so suffice it to say he’s unafraid of taking risks. From there, he joined the startup scene, where he’s been for the past sixteen years, building products at Zynga, and then founding Rally Health. His latest company, Alpha, is the on-demand user insights platform he wishes he always had. We sat down with Thor to talk about how he’s driving a culture of experimentation across the industry and why the future of product management belongs to everyone. This interview has been edited for readability.

Whenever I have a chance to talk to a founder, I always want to know: what is the broader bet that you’re making with your company?

Our company is staked on the idea that everybody in an organization should be able to generate and manage data, and use it to inform decisions. So our broader bet is that companies are going to be less centralized in terms of management structures and digital product initiatives – that people that are closest to the problem or the user will be the ones responsible for corresponding decisions. And, if those people don’t have data, they’re going to be in trouble.

Today’s method of “data-driven decision-making” is fancy PowerPoints from research and strategy firms, and that just doesn’t work. It’s slow, expensive, doesn’t scale, and generally doesn’t even trickle down to the people that would theoretically need it. We’re using technology to address this problem.

By integrating and streamlining the many manual steps involved in experimenting and putting data behind decisions, we’re enabling any company to do what is necessary to stay relevant – to be customer-centric. And it’s not just a big legacy company problem, it’s a problem for every company. Even new entrants like Snapchat, that just a couple years ago were disrupting media, are now themselves being disrupted. They need to be move faster to meet customer expectations as those preferences evolve.

How should enterprise companies be building out their product management organizations to get a little more structure?

The industry has matured significantly since we started building Alpha less than four years ago. It’s getting more common for companies, like Google, to have Associate Product Manager programs that include rotational training. That can solve the skills gap problem, which is fine, and is the problem everyone is trying to solve today. But tomorrow’s problem will be, “alright, we have enough people that are trained, but now they need the tools to do their job.”

What are those tools?

To gain a real competitive advantage with rapid experimentation, you need several capabilities that scale and work even in highly bureaucratic and regulated environments. Among those are the ability to target an audience to find whoever you need to talk to and get feedback and insights from.

You also need the ability to put the right stimulus, prototype, or product in front of that target audience. As we all know, you can’t just ask people what they want. You have to show them something to react to. It needs to look real so that the feedback is authentic, and it needs to be compared to alternative offerings and reactions so that you can analyze and optimize results. Consumers have more than one buying option in the real world, so you need to be able to validate as best you can that your offering is better than other offerings, and not just better than nothing.

Then you need to be able to analyze results objectively, and then rinse and repeat quickly. Product management is all about solving problems, and the best process for doing that is through continuous iteration. This is how the best folks in other industries do it as well. Think of a stand-up comedian telling a joke at a small local club, seeing if it works, and adjusting his routine the next time out. Lawyers set up mock trials to test arguments. Publishers use inserts in current magazine issues to test cover designs for future issues. That’s product management in a nutshell.

But there’s a big difference between doing it once and doing it at scale. Anyone can run one experiment. If you’re at Google or some other Fortune 100 company though and you’re trying to launch a product, it’s a totally different challenge. Your training is still the same, but the tools need to be able to keep up.

What are the technical skills and technical background required of a PM?

You have to have empathy. That’s probably the most important skill you need. If you’re dealing with developers, product managers have to understand the developer process, though they certainly don’t have to be developers themselves. Same thing goes for if you’re in healthcare and trying to build a healthcare product. You need to understand what the life of a doctor is. Technical skills are useful to the extent that they help a PM understand their product teams or their customers.

What have you found surprising in working with large organizations?

I think it’s important not to think of organizations as monolithic entities. There’s no such thing as an organization that collectively does things a single way. It just doesn’t exist. Citi, Pfizer, Google, etc. are all just logos. That may be who pays the bills or whose name is on your W2 pay stub, but the company is the diverse people that work there. The unit that you can impact, or that we impact, are teams. We look at an organization as really just a team of teams. They may not always be working together or the same way.

When we started, we thought that people would share data and that there would be like this inherent viral interest and growth within the company. Somebody starts getting data, understands it, shares it, and loves it. That’s true, but people tend to share it very carefully. Mostly to their higher ups, when they’re doing presentations asking for more budget. Rather than sharing it throughout the organization, we tend to see them keep Alpha as their secret weapon. They compete with other teams and they rarely want to share the actual data with other teams. That’s been a major learning for us about corporate culture.

What do you view as the interplay between qualitative and quantitative data?

When you look at your data in Mixpanel for example, you see a user journey, you see certain paths, certain numbers, certain analytics on what people are doing. But, you don’t necessarily know why they’re doing it, so you have to complement that with qualitative insights.

Until you actually sit and watch your users, you can’t feel what it means to fall out of the funnel, or churn or whatever problem you’re working on. The metrics may be the impetus to change something, but the learning, and the product management if you will, is really on the qualitative side.

How can Fortune 100 companies use their scale to be innovative?

Ultimately, innovation is about taking shots on goal, so large companies have the resources to take more shots and afford more misses. They can afford to run more exhaustive experiments, like setting up 100,000 square foot labs to test a retail store or hotel lobby layout. Sometimes they call these places innovation labs or experience centers, but really they’re an executive’s vision of what the future will look like.

There’s value in being able to actually see and touch what is possible rather than having to just imagine it. It puts a company in the mindset of figuring out how it’s going to compete in that future. To get from here to there, there are thousands of assumptions and hypotheses, each of which can be tested. It allows them to prioritize certain problems and understand the value propositions that come with those choices.

Can you talk about the differences between product management teams at smaller vs. larger companies?

Smaller and younger companies’ product teams are often better-organized and more data-driven. It’s easier for them to move quickly because they have less risk, fewer internal hurdles and established processes, fewer legacy requirements, fewer people that have been there for 30 years. Generally speaking, there is less of what slows down iteration cycles.

That’s important because moving fast is critical. I’ll give you an example. Six months ago, a bank used Alpha to test whether or not people would use facial recognition technology for authentication to log in to their bank account. Almost everyone they tested against thought it was crazy, unsecure, and even silly. Since that test, Apple famously launched FaceID on the iPhone X. The bank ran the test again and now way more consumers say they’re comfortable with facial recognition. That’s an example of an external factor influencing consumer preferences. If you don’t have a way to continuously measure and benchmark those preferences, you can fall behind trends quickly and miss opportunities.

At big companies, the best innovation I’m seeing takes place in emerging, but not necessarily bleeding-edge, technologies. Think of something like over-the-top streaming, where there’s a proven market with offerings from Netflix and Hulu. There is a market for streaming, but it’s not yet mature or saturated. For initiatives like that, large organizations are better at giving teams a decent amount of leeway to experiment and iterate quickly.

I’m a bit more skeptical of the teams working on brand new technologies, such as AI and blockchain, because they have to constantly explain what they’re working on. That can distract them from being able to do any actual work.

What are some commonalities you see in your Fortune 100 clients?

They’re all looking at how their offerings need to change so they can engage existing customers and target new customers. So we see a lot of ‘how do we sell [fill in the blank] to Millennials?’ Often that means figuring out how to build a new user experience for selling life insurance, signing up for a credit card, or buying a house. Other clients start with a technology and try to work toward potential use cases. Like chat bots or wearables.

No matter which approach they take, they need to outline their questions, hypotheses, and assumptions, turn them into data and then drive a process with their entire team from brainstorming all the way to reporting the outcomes. The faster they go through that, end to end, the more they produce and the more successful they become.

The most innovative companies achieve one or two week iteration cycles. Historically, the standard – besides doing nothing – is maybe two to three month iteration cycles. That’s commonly what our clients were doing before coming to us, and our platform enables them to achieve one or two day iterations. That’s unheard of. The bottlenecks then change from being able to generate insights to being able to schedule meetings, communicate insights, and make decisions. We’ve actually seen how that slows product teams down more than getting the data in the first place.

One last thing: finding the right leadership for the team is critical to success. When I say leadership, I don’t mean the CEO, I mean an individual team’s leader. That could be a mid-level manager. It doesn’t have to be a senior executive. The right manager can give an individual contributor the space to absolutely flourish. We do our best to highlight these leaders and their best practices across our channels and educational resources. Many readers might be familiar with our podcast, This is Product Management, or Medium publication, Product Management Insider. We just launched a new channel that has exclusive videos and resources for product executives.

What’s something you believe more strongly than just about anyone else?

What we call product management today is going to effectively go away and it will become everybody’s job. I recently published an article about that and a few other predictions.

How will it be everybody’s job? How is it going to be person in sales’ job? How is it going to be be the person in marketing’s job?

It very much is because everyone’s job is to make the customer happy. Partly that is understanding customers’ needs, which entails questions, hypotheses, and assumptions, testing them, validating them, and iteratively working through them.

A salesperson actually does the same thing. They work off a script, sure, but they’re not robots reading it, they’re going to have to be able to riff off of what a customer says, and refine their approach. I’ll go back to the stand-up comedian comparison. It’s all just iterating and improving. The ones who are the best at that become the most successful. It’s the same in every field.

You wouldn’t necessarily think to call a salesperson, designer, or engineer a product manager, or to call a stand up comedian a product manager. But what they do – run experiments and iterate – is what we call product management.

Get the latest from Mixpanel
This field is required.