How is retention calculated?
The retention calculation in Mixpanel is designed to create a normalized, actionable metric to assess user engagement over arbitrary periods of time. It is our implementation of what is known as "cohort analysis." Cohort analysis involves breaking customers apart into groups (or cohorts) based on when they completed an action. In Mixpanel this translates into grouping based on when that customer sent a certain event.
Mixpanel can cohortize (break users into groups) based on any single event, with time granularity of days, weeks or months. Or, if you select the "anything" parameter, we form a cohort of users who sent at least one event of any type during the given time period. If a user sent the event during the selected time window (you will see the windows running along the left side of the report), the user will be included in that cohort. The count of users for any cohort can be seen along the left next to the date we are cohortizing on.
We only count unique users in cohorts; not totals. In other words the cohort will include every unique user that sent that cohortizing event in that time window, starting and stopping at 0:00 of the beginning day and midnight of the ending day. A customer can only be counted once per cohort, but can be included in more than one cohort. For example, if you are cohortizing based on your "item purchased" event and creating weekly cohorts, a customer who purchased at least one item each week will be in every cohort, not just the cohort for their first purchase.
For each cohort, the following buckets (marked 1,2,3 etc. along the top of the retention report) are dependent on when any given customer sent the cohortizing event. Let's stick with the weekly "item purchased" cohort example. And say we are looking at people coming back and sending any event whatsoever (using the "anything" param). Let's say we are still looking at weekly cohorts. In that case, the numbers marked 1,2,3 etc. along the top of the report will be marking week long time buckets. The percentage under each is the number of customers who came back to send any event during that bucket.
Now listen up, this is the complicated part: the beginning and ending of each bucket will be different for each customer in the cohort. The limits of the time bucket marked "1" will be between 1 and 2 time periods after the cohorting event was sent. If a customer sends an event at 3pm on Tuesday then bucket "1" for him begins 3pm Tuesday of the following week. That customers bucket 1 extends between 7 and 14 days from the time of the cohortizing event.
Likewise, in daily retention, in order to be counted in the bucket marked 1, the customer must send whatever event we are looking for in the retention some time between 24 and 48 hours after he sent the cohortizing event. If he sends the event between 0 and 1 time periods after the cohortizing event that event is not reflected in the report.
Thus, the buckets are actually a little further forward in time that you might think, and can take up to nearly two full lengths of time (days, weeks, months) to settle out completely, since some users bins can extend nearly two units in the future.
Mixpanel offers the ability to segment your retention report by the properties sent along with your events. Using this feature you can see how many users with a specific property value are retained compared to other users with a different property value. Put simply, you can compare the retention of users based on their operating system, country, or any other property you send along to Mixpanel.
Looking at the report, you will see the number of users in each cohort will not be the same once you have segmented. This is because the calculations for cohorts and their retention are averaged in this report. What this means is if you select a date range of 6 months and choose to view monthly retention, the number of users in the cohort and their retention will be averaged across each of the 6 months. Thus, the cohorts you are viewing are the average number of users with a given property value in the time unit specified over the date range chosen.
Types of Retention
Depending on what cohorts you are trying to create, Mixpanel offers three different ways in which you can establish cohorts and view their retention. Each of these reports uses the same methods of calculation described above but allows you to specify which event and property combination performed is creating the cohorts and measuring your user retention.
Recurring retention allows you to specify a single event and see the retention of users coming back to perform this same event over time. You also have the option to choose “anything”, which will look at users who performed any event for creating the cohorts and calculating retention. This report can tell you how often users come back and perform the same events.
First time retention allows you to specify one event to create the cohorts, and another event to measure the retention on. Similar to recurring retention, you also have the option to choose “anything” as the event. This report is useful in seeing for users who performed a specific event (such as sign-up) how often they come back and do some other action on your site.
Addiction retention allows you to specify an event and see how often a user performed that same event in the future. Whereas the above retention calculations will return the number of users who performed an event, addiction will actually calculate the number of times each user who came back performed the event. This allows you to see how "active" your users are over the course of a given time period.
Using the addiction report will allow you to see how many hours in a day, days in week, or weeks in a month your retained users are performing a specific event. As with the other reports, you also have the option to choose “anything” to see generally how often users are performing events.