Company

Deliveroo, an online food delivery company based in London, England, operates in over 500 cities worldwide. By offering fast, reliable and trackable delivery, the company has seen rapid growth year on year, with partner restaurants seeing revenue increase by up to 30%.

Deliveroo operates three different groups based on user segments: Restaurants (i.e., delivery), Rider, and Consumer. Compared to other delivery companies, which do logistics really well and apply that to different domains, Deliveroo’s mission statement is bolder: to be the definitive food company. “That means we’ll take on challenges and opportunities that others might not,” says Jordan Ng, Restaurants Group, whose customers are the restaurant partners and convenience stores that sell products on Deliveroo. “Our mission allows us to explore and do things like our Editions kitchens and procurement services where restaurant partners can purchase supplies through us and leverage our aggregated buying power. What excites me is that there’s a lot of diversity within that mission statement, but it remains focused on food, which is the company’s passion.”

Jordan states that data is crucial for helping Deliveroo figure out where to direct its product teams. Without a solid data source, they can’t fully understand which direction will have the most impact.

Challenge

Deliveroo faced two major challenges.

First, the company needed to unify its fragmented analytics ecosystem. Partners could interact with multiple applications, including a marketing site, a blog, a restaurant portal, and a tablet app—all of which published metrics and events in vastly different ways.

Second, Deliveroo has a team of data scientists, but the company wanted to be able to use them more effectively and efficiently and to provide an easy way for more teams—like marketing, product and engineering—to have quick access to the data and run reports without needing to write SQL or wait for a data engineer to have time. To get all of the data from their different sources into a single funnel and to make data available to everyone in the company in a more accessible manner, Deliveroo chose to implement Mixpanel. Mixpanel also allows Deliveroo to create self-serve reports that look at engagement or how a new feature is doing—and allows data scientists to do more strategic work.

Solution

Jordan’s team focuses on small and medium businesses—the mom and pop fish shop around the corner. These restaurants engage primarily through a tool called “Restaurant Hub,” where they can do things like review operational performance and access self-serve tools to grow and manage their business.

The main metrics the SMB team focuses on are engagement (Daily Active Users (DAU), stickiness, retention), conversion and churn.

Deliveroo wants to ensure its restaurant partners are successful, that they get orders and high ROI from using the company’s services. Deliveroo looks closely at the conversion funnel from sign up to the first order. For each market, they have targets they want partners to hit, so they know they are reducing churn and increasing conversion. They track how well their restaurant partners are doing on the platform. Are they engaged? Are they taking actions that are good indicators of whether they’re going to be successful? How long has it taken them to onboard? “We want our partners to be able to go from making the decision to be on Deliveroo to actually taking their first order in a matter of hours,” says Jordan, “and being able to look at these metrics is what’s going to enable us to do that.”

Deliveroo uses Mixpanel Funnels to track user behavior, Retention reports to understand overall impact and Insights to calculate DAU.

Results

Testing a hypothesis. Deliveroo used Mixpanel to easily and quickly test hypotheses around a spike in DAU traffic. The spike occurred a few weeks after the company first implemented Mixpanel. “We noticed that we were getting a recurring trend of a spike in daily active users every couple of days, and then it would go back down to our regular traffic,” says Jordan.

Until this point, the company hadn’t examined engagement or traffic in depth, so they didn’t know how to understand spike patterns. They hypothesized that perhaps it correlated with payday—that the spike was happening as a result of us paying our partners, as they logged in to download their invoices and various other actions related to getting paid.

“We wanted to test this hypothesis, so we quickly threw events in Mixpanel and ran an Insights report that compared DAU to actions we expected people to take off the back of getting paid (such as downloading an invoice, etc.). We looked at this for a couple of weeks, but saw no correlation whatsoever.” The company continued to collect a month’s worth of data, and realized that DAU was spiking every Monday—or, in the case of a long weekend—Tuesday. “It’s the start of the work week and of course our partners are logging in. They’re looking at their feedback, their sales of the previous week and they’re making tweaks to their menu, or changes to their hours. With Mixpanel, we were able to test and disprove our hypothesis of it being related to payday.”

Without Mixpanel, Deliveroo might not have explored the spike further because that would have had to pull in a data scientist. “But with a handful of clicks in Mixpanel, we were able to put this together, put it up on our dashboard and track it week over week passively.”

Restaurant engagement. There are particular sections in the Deliveroo Hub where restaurants can look at their order and sales metrics. Anecdotally, Deliveroo expected that businesses would be looking at those infrequently—maybe week or monthly. But what they saw is a sizable selection of their partners looking at those metrics daily, if not hourly.

“That was a surprise to us,” says Jordan. “We thought that those metrics were there for end of week reporting, end of month reporting, end of quarter reporting—not something you’re going to want to look at and engage with on a much more frequent basis.”

This finding was helpful to Deliveroo because it showed them that, since restaurants were already using the feature, perhaps they didn’t need to invest more in it.

“We were debating at that time whether or not we were landing customers on the right page or not,” says Jordan. “Seeing that level of engagement reaffirmed that this is the right place to send them.”

Favorite features

• Group analytics. Deliveroo’s Restaurants group is a B2B section within a B2C company, focusing on the business entity, not necessarily the user. In many cases within SMB, it’s analogous to one user, one restaurant. Broadly speaking, that’s not the case for Deliveroo: a lot of restaurants, especially a lot of the bigger chains, have multiple users. “We don’t necessarily care whether a specific user has taken a certain action—we care whether a restaurant has taken that action. Mixpanel’s function allows us to track this.”Group Analytics also allows Deliveroo to aggregate multiple users into that restaurant level and then take a look at all of them in Funnels and create cohorts and look at retention. The company started using this feature to perform even higher-level grouping, such as the performance of restaurants under a brand umbrella.“Using Group Analytics means we don’t have to worry about employee churn impacting our metrics,” says Jordan. “If employee A is the only person logging into our tools on behalf of a restaurant and they leave, we don’t want all of the metrics that we’ve been tracking for that restaurant tag to that user. We want to tag to the restaurant. Group Analytics allows us to do that.”Almost all of Deliveroo’s reports look exclusively at Group Analytics, though there are a few cases where they look at the user level. For example, all of the marketing site’s data is published to Mixpanel, which allows the company to look at anonymous users.“We always knew that we had a mental model for the separation between a user within a restaurant. When we were wiring up properties in Mixpanel, we had to figure out whether it was a user or group property, which really drove home that separation. The modelling and how we separate this is much more evident now.”
• ID Merge. “When we saw this feature launch, it was like Mixpanel was reading our minds,” says Jordan. “Because we have a lot of fragmented systems—users come into the marketing site via third-party technologies like Salesforce and then perhaps eventually end up as a user in the Restaurant Hub—being able to track different IDs through that journey and tie everything together is something we couldn’t historically do before Mixpanel.”The launch of the ID Merge feature has made it simple to identify a restaurant that anonymously views the Deliveroo blog and that then later logs in, or to track the entire journey of a new restaurant that goes from the marketing site as an anonymous user to a lead in Salesforce to a user in the Deliveroo system.“For us, it’s about being able to understand how we can optimise our partner’s onboarding journey and their journey to success. You may have gone through all of your onboarding and mid-boarding steps and be live on the marketplace available for orders—but not seeing success yet. Maybe you haven’t gotten your first order. Maybe you have gotten one order but got a bad rating on it and maybe we start pushing you some marketing content. Maybe we give you a recommendation to improve your packaging. Maybe we recommend that you go to our delivery packaging store to purchase some of our own packaging. Mixpanel allows us to have that full picture, being able to trace their interaction across our applications.”
• Funnels. Deliveroo looks at Funnels primarily to see drop off. “It’s where can we tighten the bolts on a particular flow,” says Jordan, “whether it’s taking an action for something like an already established user going through a small wizard, or a restaurant user logging in to create an “offer” via a multi-step process. We look at Funnels to see if there’s something that we could be doing better to improve the user experience and make sure that users actually go through and complete the creation of that offer. It gives us insight into where there might be friction and helps us narrow our focus on where to explore. Maybe we need to look deeper into the data and maybe we need to look at our errors. Maybe there’s some edge case that we’re not covering that’s preventing folks from getting to that next step.”Deliveroo also watches the larger conversion funnel that looks at the journey from first sign-up to first order. Where there is friction, they look at how the time delta between those steps can be reduced by cutting out manual processes or reducing pain points.
• Retention. Deliveroo looks at one-week and three-week retention to see if restaurants are engaging with the platform. Are they finding value in using Hub? “You don’t necessarily need to use Hub to get orders,” says Jordan. “You could get set up and then completely go on autopilot and your orders can come in. You may not necessarily need to come back to that portal, but we want to help you – to help you grow and manage your business. It’s an opportunity for us to educate you, give direction, and help you improve your business so we want to measure that engagement and retention.”

Why Deliveroo loves Mixpanel

Wide variety of teams able to access data using Mixpanel. At Deliveroo, Mixpanel is used by:

• Product teams: product managers, engineering managers, engineers, designers, and data scientists.
• Commercial teams, especially the B2B marketing team.
• Data science team and BI team (especially for those engagement metrics)
• PMs and product marketing managers (especially to measure engagement and interaction with new features)

Working with Mixpanel. “The Mixpanel account team has been fantastic to work with,” says Jordan. “When we first implemented Mixpanel, we noticed a large discrepancy between the number of login users that we had and the number of MTUs that we were recording in Mixpanel. Our Mixpanel account manager put us in touch with Support. My engineers ended up working with the support team and eventually were able to figure out and fix the root of the issue.

“The other anecdote that I would like to highlight is that we ran a training session with Mixpanel for around 40 employees across multiple different disciplines. That level of support engagement was really appreciated. It went a long way toward getting people excited about and using Mixpanel.”

General advice from Jordan Ng

“Publishing the right business metrics must be integrated into your development process. What does that mean? When I go and ask an engineer to pick up a particular feature or take on a task, I don’t need to explicitly ask them to write a unit test to test their code. It’s part of their process of developing a feature. The same thing goes for publishing operational metrics. Making sure that the service is healthy and running those metrics should not be a footnote. We need to apply this same train of thought and rigor when thinking about business metrics.

In most companies that I’ve been at, that isn’t the case. It’s not a given that that’s going to be done by default, as part of the task of building a feature. Baking this into the culture and the expectation is probably the most important and most impactful thing that you can do.

You can have all the tools in the world, but if you’re not publishing the data—or you need to go back and add it retroactively because your product manager or your data scientists didn’t think to ask of it during development and now you need to answer a particular question—you’re two steps behind where you want to be.

We need to be asking these questions: What’s the useful metric? What do we want to be measuring? Is there an event we should be publishing to Mixpanel? That’s my biggest advice: build it into the culture and build it into the development process so that it becomes a given.”