AARRR! Mixpanel for Pirates (updated!)

Dave McClure's Startup Metrics for Pirates has become something of a bible for startups focused on collecting data and iterating quickly. His premise is simple - building a startup is hard! To succeed, you need to design and test constantly throughout both marketing and product. Marketers must design and test multiple channels, measuring effectiveness deep down the funnel rather than on the surface. Product designers should rigorously test new features through split testing and measuring impact on conversion.

For founders, less is more; the entire team should be focused on five key metrics: Acquisition, Activation, Retention, Referral, and Revenue.

Luckily for all of you pirates out there, Mixpanel is the perfect tool for tracking AARRR....

Acquisition

The key to understanding user acquisition is understanding which ad networks, widgets, campaigns, affiliates or SEO/SEM are sending you valuable traffic. Dave McClure calls these "best-performing conversion channels" and emphasizes that these are not channels sending you tons of pageviews, but channels sending you users that do something meaningful. This might mean viewing a movie, downloading a plug-in, playing a game, entering an email address - it's really up to you. All that matters is that you determine your meaningful events and judge your sources of traffic accordingly.

Mixpanel lets you easily track these meaningful events and segment them by source, campaign, medium, keyword, etc. Here is a sample Segmentation report. We are digging into an event that is more meaningful than a "pageview" - we're looking at a Browse, or search, event. Most likely, users that hit my landing page and actually browsed for an item are a good audience for me, and are much more likely to use my product than visitors who bounced after hitting the site. Breaking this out by ad source, we can see that HeyZap is the best source of traffic that performs this meaningful action.

An even more sophisticated approach to judging sources of traffic would be to run your retention reports by source to see not only which campaigns are sending you valuable traffic, but which campaigns are sending you traffic that returns over and over again. This is something unique to Mixpanel; I'll get more into this under Retention.

Dave McClure also mentions audience segmentation - who are these users? What age/gender/geography do they represent? Mixpanel Segmentation lets you drill deeply down into your conversion event, asking questions like, "Of the female visitors who came from Google, live in New Mexico, and purchased a pillow, what does their age distribution look like? What keywords did they search for?" It is imperative that you know who your customers/users are! No tool out there lets you dig as deep as Mixpanel.

Activation

So you've worked hard to get these valuable users to your website by optimizing your sources of traffic. Now, you need to make sure these users have a "happy first visit." Do they move through your app or website seamlessly? Are there certain pages or steps where you are losing or stalling a chunk of your traffic? McClure suggests constant A/B testing of your landing page as well as every added feature, with the key goal of increasing conversion.

Luckily for you pirates, it's a breeze to set up conversion split tests in Mixpanel. In the screenshot below, we are testing a sign-up form to see whether our alpha test has a greater conversion rate than our baseline. We can easily see that our test version outperformed our baseline, and we could drill down even further to see the impact by particular audience types as well as the impact on median time between steps.

Retention

So far, so good - you've optimized your traffic and created a happy first touch. Now, you need to keep your users coming back! You also need to be able to tell whether the changes you're making are impacting retention in a positive or negative way. The key to monitoring changes in retention is to view it by cohort groups.

Mixpanel uniquely gives you the ability to break your retention into cohort groups (birth classes) as well as to segment it by attributes like Source, Gender, Age, City, Device, Version, etc. This way, you can see not only whether your retention is getting better over time, but you can ask questions like "Are my changes increasing retention of my target audience - women who came to my site via Facebook?"

The question I'm frequently asked is, "This is all well and good, but how do I get my retention to go up?"

Well, Mixpanel gives you a great tool for this: Engage.

Engage lets you target your audience by very specific attributes and then test to see if the users you reach out to are reacting to your emails/notifications. For example, you could target users who have made multiple purchases in the past but haven't been to your site in two weeks to let them know about an event you're hosting in San Francisco, and then use funnels to see if these notifications are driving them back to your site or - even better - whether these notifications are impacting retention.

Referral

Once your product is battle-tested and beloved, it's time for viral marketing. This is your cheapest source of traffic, and one that bodes well for the success of your business - users like your product so much, they want to share the experience with others!

In Mixpanel, an easy way to start is by tracking shares, or even generation of shares, by adding tags in the URL. For example, we could easily segment an event called "share" by a property called "social network" -

You can take this even further to track conversion of these shares. We recently wrote a blog post on Tracking Virality with Funnels, which explains in detail how to track number of invites sent and percentage of those invites converted.

Tracking conversion of shares gives you two powerful tools: you can calculate your K-factor as well as see which users are contributing to your virality. Using mixpanel.people.increment, you can tally the number of share events fired by a user. Perhaps you'd like to thank them for their support directly via Mixpanel Engage? After all, it is certainly important to focus on the retention of your biggest evangelists!

Revenue

All of the previous four metrics were really teeing up the most important metric - revenue. Your revenue model is entirely up to you, and, of course, varies with each business. That said, measuring and tracking the lifetime value of a customer is an incredibly powerful tool to understand how much you can spend to acquire those customers while remaining profitable.

When you know who your most valuable customers are, you know who to focus on and where to focus development efforts. You'll be able to see if 18-24 year old men in San Francisco or 18-24 year old women in New York spend more money in your app. You can measure which acquisition channel delivers the most valuable customers and how this changes over time. In a nutshell, you can optimize your product and your marketing around the most important metric of all - dollars.

So...are YOU a pirate navigating formidable seas? Mixpanel can get you to your treasure. Ahoy, mates!

Why most A/B tests give you bullshit results

By now, anyone in product or marketing knows what A/B testing is. What we don’t know, or at least won’t admit, is that too many A/B tests yield nothing. Too often they measure meaningless variants, produce inconclusive results, and nothing comes from them. Of course, some A/B tests yield real, meaningful, actionable results. Those are the ones you hear about. We've all seen the articles. Company X increases conversions 38% with this simple trick. Hell, I've written some of them. But those success stories have hidden the grey underbelly of testing and experimentation. AppSumo revealed that only 1 out of 8 tests produce results . Kaiser Fung estimates that 80 to 90 percent of the A/B tests he’s run yield statistically insignificant results . Yet many new testers walk into A/B testing thinking it'll be quick and easy to get results. After running a handful of simple tests, ...

Announcing Mobile Dashboards for iPhone

We've launched our Mixpanel mobile app so you can monitor your key metrics in one place, everywhere you go. With our iOS app, you can stay up to date on your company's trends, conversion rates, and retention all in one place. Available free in the App Store. Everywhere we look , the data shows that mobile is taking over . So we're putting the world's most powerful analytics into your pocket with Mixpanel's Mobile Dashboard app. Download our app for free , and log in with your Mixpanel account. Mixpanel's mobile app allows you to create a dashboard of your most important metrics, so you can easily check-in on your business anywhere you go. By easily adding key metrics to your dashboard, your data is available instantly for when you need it on the fly. When you need important stats for your team's daily standup meeting, during drinks with an investor, or wherever ...

Re-engage your users based on what they do, not just who they are

Mixpanel People is a powerful way to connect with your users and drive engagement. Now you can send notifications to your users based on all of the events you already send to Mixpanel. Available now for Push, Email, SMS, and Webhook notifications. Send targeted notifications to your users With Mixpanel’s People platform, you can target users based on the actions they take in your application. Here are some examples of how you can use this: Send a push if a user exercised 5 times in their first 30 days Send an email if a user has only logged in 1 time in the last 7 days Send a SMS if a user hasn’t chatted with another user yet Easy to Use It is extremely easy to use this functionality: you can retroactively use all the events that you send to Mixpanel. Start by going to the Notifications tab , then selecting your notification type and notification message. Now i...

Downtime: Ingestion API post-mortem

On January 20th, 4:05pm PST, Mixpanel’s ingestion API was intermittently unavailable until approximately 8:55pm PST. This incident was caused by two main factors: insufficient capacity in our backend queuing system and a bug in our Android SDK that produced a thundering herd . At 2:30pm, we rerouted traffic away from one of our datacenters due to planned network maintenance, placing double the load on our Washington, DC datacenter. At 4:00pm, a 10X spike in ingestion traffic occurred. Combined with the additional load from the maintenance, we started queuing data. We queued enough data to cause our queue servers to become disk IO bound. This lowered the enqueue throughput, causing some API requests to time out. Due to a bug, our Android client library responded to these timeouts by retrying every second instead of backing off. This triggered a thundering herd. The influx of retr...

How Breather built a network of spaces and succeeded where so many startups fail

Supplying peace and quiet while solving the on-demand economy. “Acquire users.” That's the common wisdom for consumer-facing startups. The more users, the better. Not unexpectedly, many of these companies focus all of their energy on driving demand, and often it makes sense. Just look at the tech giants that have built their businesses on a foundation of millions and millions of users, often acquiring masses before they even knew what their businesses really was. And if you're Facebook or LinkedIn, it makes sense. These companies aren't worried about supply. What are they going to do, run out of LinkedIn? (Ignore the dirty looks Ops folks are sending my way.) In fact, because of demand-side economies of scale and the network effect , LinkedIn's product actually gets better with more users. But there's a new breed of startups playing a different – and often lucrative – game. T...

A guide towards App Store Optimization

Let’s face it, getting your application discovered in the App Store or Google Play is a huge challenge but if you find a way to crack the algorithm, the payoff is huge! There are more than 1.5 million applications on the Apple App Store that have been downloaded more than 100 billion times. With the average number of downloads an app gets being around 62,000, App Store Optimization ends up being an important technique that can push you way past the average. What is App Store Optimization? App Store Optimization is a technique app makers use to improve the visibility of a mobile app in an app store. It’s similar to search engine optimization (SEO) for websites but optimizes for discoverability for a mobile app. The basic techniques of ASO encompass doing good keyword research, improving your app description, utilizing high quality screenshots, doing competitive research, and...

Why QuizUp turned the fastest-growing game in history into a social platform

Two years ago, QuizUp became the fastest growing game in the history of the App Store. But to stay relevant and to maintain its users, they knew it had to become more. With data-driven development, QuizUp has grown into an app that brings its users back time and time again, transforming from a trivia game into a social platform. Every week, somewhere in the ballpark of 15 thousand apps are submitted to the iOS App Store. About a quarter of those are games. A quick glance at the Featured screen shows a "Best New Games" list with a couple dozen games released in the past week or so. They're from names as big as Lego, and as small as Futureplay, a five-person shop out of Helsinki releasing their first game. It's a highly competitive landscape. And each can only dream of the success that Plain Vanilla Games had in the first week with QuizUp: over a million downloads. Five months after ...

Demystifying Predictive Analytics for Mobile and Web Apps

Predictive analytics is unfathomable to many of us. Terms like artificial intelligence, neural networks and deep learning just deepen the mystery. Yet predictive tools are already ubiquitous. Not in self-driving cars, or robots that look like humans. Rather, on your phone. Spotify looks at your past listening habits, and that of people who like similar music, and recommends new tracks to listen to. Youtube does the same with videos. Your email service predicts whether a mail message sent to you is spam by analyzing past emails sent to you and others. Siri and Google Now use predictive technology to understand your voice, and offer answers to your questions. Predictive analytics is the practice of using advanced statistics and historical data to predict future outcomes. For the most part, the use of predictive analytics in business is still shrouded in mystery and the domain of ...

Introducing Predict - see who will convert, before they do.

Predict lets you know which of your users are going to convert, or not, before they do. Just select an action and, based on users' previous behavior, you'll get a grade on how likely each user is to complete it. Then target users by their grade. Like reaching out to those less likely to convert and giving them a nudge in the right direction. And starting today, Mixpanel Predict is live for all customers using People Analytics . Use Predict to impact leading metrics Use Mixpanel's predictive analytics to learn whether your app’s users will convert. You can define what converting means for your company as any action or event in Mixpanel – even the number of times it was done. Pick an important action that leads to success for your app (commonly called a leading metric ). So if you have a photo sharing app, you probably want to know who is going to share a picture. Or who wo...