Visualizing your customer’s support experience through Mixpanel

In 2021, we started reviewing Mixpanel support ticket metrics with … Mixpanel. Here’s an outline of how we approached it, our reasoning for why we wanted to do it, and the value we are getting out of the implementation.

Any analytics implementation should start with the set of North Star metrics that are relevant for the company and departments. In Mixpanel’s support team, as it happens in nearly all organizations, we have many goals at a given point in time, but with improving customer experience as the theme, as well as for simplification purposes, we focused on three strategic metrics and three operational ones.

Note: This is meant to be shared with the product team to help drive prioritizations. Ticket volume on its own could be a lead indicator. That being said, it can be weighted by revenue and ticket urgency/impact, instead of just sheer ticket volume, in order to review the categories that have could have the highest impact.

Note: Both weekly/monthly volumes and escalation times are used for capacity planning while “my” daily tickets is more of an individual metric that gives each support engineer their own dashboard with the volume of types of tickets they are working on.

The metrics need to be translated into specific events and properties to be tracked. A sample implementation layout for those metrics above can be found in this link.

Before continuing with how to implement, it’s important to state the motivation in visualizing the metrics through Mixpanel since, at a glance, it might feel like re-inventing the wheel. Zendesk does, for instance, offer a very capable reporting offering and Mixpanel is mainly geared towards product analytics. This could be summarized into two reasons.

We wanted to make sure we could analyze the metrics that mattered the most to our team. One, in particular, was the p90 on (FRT). Zendesk, of course, already provides a way to review the FRT for a ticket, as well as the median and averages. For quite a while, we measured average FRT as part of our metrics, but a challenge that arose was that while the average was within a given predictable range, a non-trivial number of requests were being replied to quicker than the average, while others took quite a bit longer. This meant that the average was skewed at both ends and not necessarily representative of the overall customer experience.

In order to reduce this noise and make sure we kept an eye on a more comprehensive view of the experience for response times, we decided to shift from median and average reply times to measuring the p90. The goal would be to measure what was the response time under which 90% of all incoming requests for a period of time were being answered. Luckily, measuring percentiles (p25, median, p75, p90, and p99) is already baked into Mixpanel.

The image above tells the story: The orange line represents the median FRT, the purple one represents p90, and the gap in between represents the difference in wait times between half the requests vs the top 10% that take the longest to reply to. As time progresses, we measure and adjusted with multiple initiatives. As a result, we can see the trend over time of the overall customer experience getting closer within most of the spectrum.

Having insights on response times, resolution times, and NPS are great tools to have within the support team to better the experience we provide. But in order to maximize the impact on customer experience, ideally, we want to empower the product team with data on trends from customer feedback or just sheer volume counts to assist in the planning and prioritization process. Said in a different way, we want the product and accounts team to be informed on why customers are writing in.

Since everyone at the company already knows how to use our product, how to set up dashboards, and how to do explorative analysis on datasets using our reports, a way to minimize friction and promote the use of the data is to place it in the tool everyone on our end already knows and understands within the company.

There are a number of ways to make this happen, and we will cover a minimal-tools-required approach, but essentially, the best way to proceed would be dependent on your data ecosystem. As an example, some organizations may already have their Zendesk data in a data warehouse (DW) in order to be able to cross-reference with other internal data. In that case, a great strategy is to start from that centralized data in your DW and leverage a tool like Census that can sync it to your Mixpanel project.

As a general idea, having your data centralized and cleaned in a DW and then pushing it to Mixpanel for visualization is a great way to go.

Having covered that, the approach we want to cover here assumes no additional tools and minimal infrastructure required. The layout of the implementation would be:

Covering points 1 and 2 would get us most of our metrics and would directly be sent from Zendesk. The gap they have is FRT and full response time, as those values would not be available in the trigger. That being said, since this is a metric we don’t need real-time access on for dashboards, the script in step 3 can take care of extracting this data, say, once a day. You can have this script running in a local machine or even just publish it to a cloud platform like Google Cloud Platform or AWS on something like a cloud function.

For a much more detailed review of the steps, we’ve laid out all the details in this GitHub repository.

We didn’t want to close the conversation without covering privacy. It’s more than a cliché by now, but it holds true that with great power comes great responsibility (a quote from Marvel Comics). Given that Zendesk can potentially hold sensitive/private data from our customers, we want to make sure that the design of data coming in only includes the general data information we need for our metrics and no more.

A few tips/best practices would be:

Happy analyzing!