AARRR! Mixpanel for Pirates (updated!)

Dave McClure's Startup Metrics for Pirates has become something of a bible for startups focused on collecting data and iterating quickly. His premise is simple - building a startup is hard! To succeed, you need to design and test constantly throughout both marketing and product. Marketers must design and test multiple channels, measuring effectiveness deep down the funnel rather than on the surface. Product designers should rigorously test new features through split testing and measuring impact on conversion.

For founders, less is more; the entire team should be focused on five key metrics: Acquisition, Activation, Retention, Referral, and Revenue.

Luckily for all of you pirates out there, Mixpanel is the perfect tool for tracking AARRR....

Acquisition

The key to understanding user acquisition is understanding which ad networks, widgets, campaigns, affiliates or SEO/SEM are sending you valuable traffic. Dave McClure calls these "best-performing conversion channels" and emphasizes that these are not channels sending you tons of pageviews, but channels sending you users that do something meaningful. This might mean viewing a movie, downloading a plug-in, playing a game, entering an email address - it's really up to you. All that matters is that you determine your meaningful events and judge your sources of traffic accordingly.

Mixpanel lets you easily track these meaningful events and segment them by source, campaign, medium, keyword, etc. Here is a sample Segmentation report. We are digging into an event that is more meaningful than a "pageview" - we're looking at a Browse, or search, event. Most likely, users that hit my landing page and actually browsed for an item are a good audience for me, and are much more likely to use my product than visitors who bounced after hitting the site. Breaking this out by ad source, we can see that HeyZap is the best source of traffic that performs this meaningful action.

An even more sophisticated approach to judging sources of traffic would be to run your retention reports by source to see not only which campaigns are sending you valuable traffic, but which campaigns are sending you traffic that returns over and over again. This is something unique to Mixpanel; I'll get more into this under Retention.

Dave McClure also mentions audience segmentation - who are these users? What age/gender/geography do they represent? Mixpanel Segmentation lets you drill deeply down into your conversion event, asking questions like, "Of the female visitors who came from Google, live in New Mexico, and purchased a pillow, what does their age distribution look like? What keywords did they search for?" It is imperative that you know who your customers/users are! No tool out there lets you dig as deep as Mixpanel.

Activation

So you've worked hard to get these valuable users to your website by optimizing your sources of traffic. Now, you need to make sure these users have a "happy first visit." Do they move through your app or website seamlessly? Are there certain pages or steps where you are losing or stalling a chunk of your traffic? McClure suggests constant A/B testing of your landing page as well as every added feature, with the key goal of increasing conversion.

Luckily for you pirates, it's a breeze to set up conversion split tests in Mixpanel. In the screenshot below, we are testing a sign-up form to see whether our alpha test has a greater conversion rate than our baseline. We can easily see that our test version outperformed our baseline, and we could drill down even further to see the impact by particular audience types as well as the impact on median time between steps.

Retention

So far, so good - you've optimized your traffic and created a happy first touch. Now, you need to keep your users coming back! You also need to be able to tell whether the changes you're making are impacting retention in a positive or negative way. The key to monitoring changes in retention is to view it by cohort groups.

Mixpanel uniquely gives you the ability to break your retention into cohort groups (birth classes) as well as to segment it by attributes like Source, Gender, Age, City, Device, Version, etc. This way, you can see not only whether your retention is getting better over time, but you can ask questions like "Are my changes increasing retention of my target audience - women who came to my site via Facebook?"

The question I'm frequently asked is, "This is all well and good, but how do I get my retention to go up?"

Well, Mixpanel gives you a great tool for this: Engage.

Engage lets you target your audience by very specific attributes and then test to see if the users you reach out to are reacting to your emails/notifications. For example, you could target users who have made multiple purchases in the past but haven't been to your site in two weeks to let them know about an event you're hosting in San Francisco, and then use funnels to see if these notifications are driving them back to your site or - even better - whether these notifications are impacting retention.

Referral

Once your product is battle-tested and beloved, it's time for viral marketing. This is your cheapest source of traffic, and one that bodes well for the success of your business - users like your product so much, they want to share the experience with others!

In Mixpanel, an easy way to start is by tracking shares, or even generation of shares, by adding tags in the URL. For example, we could easily segment an event called "share" by a property called "social network" -

You can take this even further to track conversion of these shares. We recently wrote a blog post on Tracking Virality with Funnels, which explains in detail how to track number of invites sent and percentage of those invites converted.

Tracking conversion of shares gives you two powerful tools: you can calculate your K-factor as well as see which users are contributing to your virality. Using mixpanel.people.increment, you can tally the number of share events fired by a user. Perhaps you'd like to thank them for their support directly via Mixpanel Engage? After all, it is certainly important to focus on the retention of your biggest evangelists!

Revenue

All of the previous four metrics were really teeing up the most important metric - revenue. Your revenue model is entirely up to you, and, of course, varies with each business. That said, measuring and tracking the lifetime value of a customer is an incredibly powerful tool to understand how much you can spend to acquire those customers while remaining profitable.

When you know who your most valuable customers are, you know who to focus on and where to focus development efforts. You'll be able to see if 18-24 year old men in San Francisco or 18-24 year old women in New York spend more money in your app. You can measure which acquisition channel delivers the most valuable customers and how this changes over time. In a nutshell, you can optimize your product and your marketing around the most important metric of all - dollars.

So...are YOU a pirate navigating formidable seas? Mixpanel can get you to your treasure. Ahoy, mates!

How to give a killer product talk

We’re not huge on listicles over here, but we’ve been told they’re engaging, and this article is all about engagement. Not in-app, but on-stage. There may come a day when you're invited to a sensibly lit room with sensibly adult beverages (such as Mixpanel’s Office Hours) to explain to a crowd of developers, product managers, data scientists, and tech enthusiasts how you built your product. And having a 20-minute mark to make your point, you’ll want to find the best way to communicate your biggest victories. We’re going to help with that. For the past few years, we’ve had the brightest minds in tech coming through on a (roughly) monthly basis to talk about how they used data to build awesome products. We’ve seen speakers from startups, mid-market companies, enterprise businesses, and incubators. They’ve called on all sorts of multimedia, sketches, and customer feedback, but the...

Here’s what happened when Mixpanel finally built an app

“You’re going to have to shut off your computer. We’re about to take off,” said the flight attendant. Sergio, an iOS engineer for Mixpanel, needed one more minute. He was about to push his code and solve a problem in the app that had been bugging the team for weeks. Getting this update over to the team before a major product review felt just as urgent as the plane taking off on time. Mixpanel was about to launch our app and there was no time to waste. Our users can relate to that feeling on a daily basis. Building an app, especially under a deadline, is a high-pressure job. And to be completely honest? We should’ve known this already. At Mixpanel , we do analytics to help mobile developers learn from their data so they can do their job well and build something great. But here’s the kicker. We have been helping app builders do their jobs without doing it ourselves – until...

The Zen of Tomasz Tunguz

Whoever can see through all fear will always be safe. – Tao Te Ching A whisper in a Manhattan barroom, an idle conversation at a Palo Alto coffee shop, and rumor mills start to churn. The internet streamlines and aggregates this gossip and soon, before it should be possible, markets are dipping and someone halfway around the world is panicking, and the panic builds until companies are shedding value and employees. The antidote to all of this gossip and speculation is, as it should be, data. That’s what gets Tomasz Tunguz up before everyone else in the Valley. It’s why everyone from entrepreneurs to investors flock to his blog. His data-driven insights and metrics offer sanity and clarity on the state of tech, and of SaaS in particular, and cut through the panic. But that doesn’t mean his chart-filled posts don’t also have a quietly beating heart to them as well. “T...

Julie Zhou's discipline of growth

“Oh my God. How do you do that?” This is the slack-jawed question for Julie Zhou when friends learn that, after turning off the lights at Yik Yak’s West Coast office, she hits the gym and pumps 240 lbs. of pure iron. But Julie is certifiably flyweight, weighing in at just over 100 lbs. How can she lift something at that scale? Her answer, as it turns out, is fairly rote and unexciting: “Every single week, you lift one amount. Next week, you add five more pounds. Then, you add five more.” And yet there may be a deceptive genius here. Deadlifter by night, Julie effects a similar magic in her work life. She helps startups get swole. General Assembly’s Introduction to Growth Hacking is a slightly more expensive (and extensive) version of Julie’s answer. At the helm of this class, she draws from her decade-long experience growing companies and teaches how strategic perse...

Danielle Morrill's guide to the galaxy

You can depend on Danielle Morrill, co-founder and CEO of Mattermark, to draw the line between what’s important and what’s industry bullshit. Notorious for her Tweetstorms – dropping the mic on a recent one with, “Yes, I’m ‘too aggressive’ and that’s why I get what I want” – Danielle talks like she acts: fast, often punctuated with four letter words that might make your grandmother blush. I witnessed her electric personality in full force when I met Danielle at Mattermark for this interview. I was surprised when she didn’t take me to one of those airy, glass-plated conference rooms that litter most “open plan” startup offices. Instead, she led me downstairs, walked up to a bookcase, and tugged on a copy of The Hitchhiker’s Guide to the Galaxy, opening the door to “The Secret Office.” “This room gives us major nerd cred,” she told me with a grin. It’s also where she presents j...

Introducing JQL: A Query Language for Analytics

Today, we are excited to announce the official release of JQL (JavaScript Query Language), a powerful new way to access your Mixpanel data. By letting you write queries using JavaScript, JQL is flexible enough to answer any question about your data while remaining familiar and intuitive. Although JQL is a general-purpose tool, it is designed to make it easy to express typical analytics queries about customer behavior. It has a functional programming design, centered around streaming primitives like map, groupBy and reduce. By composing these elements, it is easy to write queries that scan over user activity streams, compute aggregates, or slice and dice the dataset on multiple dimensions. The aim of this post is to explain the purpose of JQL, the motivations behind its design, and our plans for its future. Why create JQL? We created JQL so that our customers could query their...

Why tracking less got VSCO more

In a meeting room in the VSCO office, on the corner of Broadway in Oakland, Steven Tang and Matt Turner stood in front of a whiteboard that was covered in little neon colored squares. Each Post-it Note had a phrase scribbled across it: “Picture taken,” “Filter applied,” “Image saved.” There were tons of them, accounting for each and every event that VSCO was tracking in their photography app. Some were vital to evaluate the success of the app, but many weren’t. It was an illustration of a problem they already knew: their data had gotten out of control. Matt is a Product Manager at VSCO, an app that enables over 30 million active users to create, discover, and connect through images and words. He has been there for four years and is part of the core team driving growth and engagement. He is now responsible for metrics across the company. Steven is an iOS Engineer at VSCO an...

No good data goes unpunished

Every so often, an email will appear in the inbox of careers@bayesimpact.org: “I have a good job. I’m ostensibly successful. But I feel empty inside.” The recipient of these emails can relate. Eric Liu, Paul Duan, and Everett Wetchler founded Bayes Impact after their high-tech gigs had stopped answering meaningful questions. Since 2014, the nonprofit has recruited several full-time engineers and data scientists to its cause: solving the world’s most intractable problems through an ambitious mix of data science and software. "Not to knock anyone who gets a lot of satisfaction from working in a for-profit job,” Everett begins. “I worked for Google. I worked with wonderful, kind, brilliant people on fascinating scientific problems.” Although he sounds sincere, one can hear the but... to follow. He too came from a good job. He too was ostensibly successful. He too was feeling...

Data Loss Incident Update

This post is an update regarding the data loss incident we had a month ago . The purpose is to describe what we are doing to prevent similar failures in the future. Incident Recap On February 22, 2016, we had a data loss incident that resulted in 9% of our customers losing their events from February 22 or 23, 2016, depending on the customer’s time zone. For more than half of those affected, we lost less than 1% of events from that day. However, in many cases it was more, and in some cases, it was the entire day’s worth of events. It was our worst instance of data loss in four years and a very stressful, disruptive experience for our affected customers, the Mixpanel engineering team, and the rest of the company. The incident raised two fundamental questions. First, how did a bug get rolled out to all production data servers without being detected? Second, given that bugs will ha...