Product analytics & enterprise organizations
This post is written by Human37, a data-driven customer experience agency located in Brussels, Belgium. Human37 helps you define, design, implement and activate your tailored and future-proof customer data strategy. They aim to shape a world that provides a better customer experience for individuals.
This article investigates the challenges larger organizations face when implementing product analytics, how to overcome them, and how to successfully develop a data-driven product development culture that focuses on providing excellent customer experiences.
The cliché that tools are only a means to an end and that success depends on what you do with them is particularly true when it comes to product analytics. The role of product analytics is to surface insights on how users are behaving while engaging with your products (apps, websites, etc). Those insights can change the future of your product and thus organization if, and only if, they are integrated into your product development cycle. Depending on the organizational type you’re in, this can either be a given or a monumental challenge.
A given when you’re operating in small organizational structures or organizations where there’s a clear focus and commitment to product development. These organizations are often centered around a product team that takes care of the end-to-end product development cycle. In enterprise-grade organizations, however, this could become a monumental challenge. These organizations have rigid structures, data silos and processes that don’t allow the insights sourced from your product analytics solutions to translate into new product updates. But why is that?
Deploying product analytics sounds self-explanatory. Generate an insight, and move it into production. In smaller structures, where product analytics and product development are two domains that are under the control of a product team, the integration happens almost naturally.
However, the luxury of this simplicity tends to be reserved for those smaller companies. Just like there is still confusion on the difference between marketing analytics and product analytics, there is still confusion in larger and more traditional organizations on who should lead product analytics. The reason for that? Your organizational structure, the distribution of competencies and skills, and the (lack of) processes.
In enterprise-level organizations, the usual structure has determined that any analytics-related topic belongs to the marketing team or the analytics team. Meanwhile, product development resides with IT. Product analytics is being looked at as just another analytics solution and therefore attributed to the teams already operating in these fields. Even though that could make sense in terms of skill set, these teams have little to no influence on the product roadmap as product roadmaps in larger organizations are usually managed by IT departments. This means that product analytics tools are used by teams to generate insights on how the product could perform better but these insights aren’t materializing into product development roadmaps, sprints or releases.
All organizations have teams focused on optimizing the product further. In some organizations, like enterprise-level blue chip organizations, they’re not called product teams. But they’re still there. In organizations with bigger IT departments, the practice of developing the product, strategy, operations and the processes and multi-disciplinary teams involved is often referred to as DevOps.
Amazon defines DevOps as:
DevOps is the combination of cultural philosophies, practices, and tools that increases an organization’s ability to deliver applications and services at high velocity: evolving and improving products at a faster pace than organizations using traditional software development and infrastructure management processes. This speed enables organizations to better serve their customers and compete more effectively in the market.
In an ideal world, the DevOps process looks like this:
- Insight generation: Using a product analytics tool to understand how users are behaving in your application, website or platform. Essentially everything explained in the first chapter.
- Hypothesis construction: Now that you have found an insight, aha moment or feature that should be focused on, it’s time to build a hypothesis. This is when you define what you believe should be done in order to drive more of the intended behavior.
- Test: Deploy the variation you’d like to test.
- Evaluate: Evaluate which variation leads to an increase in the desired behavior.
- Develop: Push the winning version to the development team’s spring backlog.
However, in many organizations, the “official” DevOps process only picks up as of the “Test” phase. Insights are barely generated or are based on gut feeling rather than based on empirical evidence (read: product analytics data).
Knowing that product analytics is typically shipped off to a team already doing some sort of analytics, this begs the question: “is product analytics then managed by the wrong team in these organizations?” Not per se. Operating a product analytics tool successfully requires a specific skill. One that is rarely present in DevOps teams. But what is the solution then?
You basically have two options:
- Create a team. Organizations that don’t have dedicated product teams could create one. Today’s IT teams in large scale organizations often apply DevOps processes but the scope they have to deal with goes beyond true customer-centric product development and optimization (security, hardware, etc). Building a product team allows an organization to truly adopt customer-centric DevOps processes with a sustained focus on driving best-in-class customer experiences.
- Adapt your processes. Alternatively, organizations can adapt their processes to build bridges between departments that hold the skill sets required to operate them (marketing, analytics) and teams that have organizational power and control over the product roadmap (IT). The result is an adaptation of the existing DevOps processes and workflows, with product analytics acting as the bridge connecting teams.
As the first option would fundamentally shuffle things around in organizations and likely be more difficult to implement, the second option is often more appealing as it can be put into production by simply focusing on process + cultural change. Let’s have a look at how the existing siloes can be bridged in order to drive the results we’re looking for.
What does such a reviewed process look like? When done well, it’s very pragmatic.
Let’s have a look at the DevOps process as shared earlier on and explore how it might come to life with product analytics integrated.
The observing reader noticed that steps 2-4 are actually steps from a traditional A/B testing process. This makes sense; commonly, we might like to test different variations of a possible UI configuration before letting the development team spend time perfecting a permanent version of the solution. What we add to this process, when integrating product analytics, is the insight generation that fuels decision-making within your experimentation program.
Conversion rate optimization (CRO), therefore, sits between product analytics and development. Or better yet, is simply an integrated part of a customer-centric DevOps process. In practicality, this means that the new process spans multiple entities within your organization and enabling different teams to work within it becomes the key to building a data-driven product culture.
How you deploy this processes and who executes what will be specific to your organization, and that’s ok! What matters is everyone agrees on the setup and is bought in to the result it is meant to drive.
Here are a few common ways we see teams aligned to our integrated approach to DevOps:
All of these variations are viable options for a customer-centric DevOps process executed by distributed teams.
While we hear it time and again, many of us choose to ignore that technology is a means to an end, not a solution in itself. But we urge enterprises looking to integrate product analytics into DevOps to place an emphasis on people and culture when rolling out these processes.
Combining technology with the human capital in your company, as well as designing the processes that allow you to leverage both of them in an optimal way, is the key to successfully building a data-driven culture–and ultimately increasing your rate of innovation.