We want to explain what happened, how we’ve worked to fix it and what we’re doing to ensure that something like this doesn’t happen again.
Before we go into these details, we want to apologize to all of our customers. None of this should have happened, and we will not make excuses for it. Our priority is to ensure that our customers’ data is secure at all times, and we are conducting a thorough investigation of what happened and how we handled this.
On January 5th, 2018, a customer informed us that they observed Autotrack sending the values of password fields in events. We confirmed that this was unexpected behavior; by design, Autotrack should not send the values of hidden and password form fields.
As stated above, our forensics and security experts have not seen any indication that this data was downloaded or accessed by any Mixpanel employee or third party. Upon discovery, we took immediate steps to secure the data and shut down further receipt. As of February 1st, all data that was inadvertently received has been destroyed. In order to be as transparent as possible, here is more detail on how we have addressed and will continue to address this issue.
How we’re addressing this issue
Since discovery, we have been actively working to resolve the issue for affected customers. Only about 4% of Mixpanel projects were impacted, and we have reached out directly to those customers with impacted projects.
We took immediate steps when we discovered this data ingestion issue in order to:
- Limit further receipt of data: On January 9th, we implemented a server-side filter to securely discard this data as soon as we receive it, and soon thereafter refined the filter to solve for the last remaining edge cases.
- Delete the inadvertently received data: We have cleared all data from our database that we inadvertently received and, upon request, have offered affected customers fine-grained metadata about what data was inadvertently sent to Mixpanel servers.
- Review any access of this data: We do not believe this data was downloaded or accessed by any Mixpanel employee or third party. To the extent we discover otherwise, we will immediately notify our customers.
In addition to fixing the root cause of this issue, we’re taking proactive steps to identify and prevent similar issues from occurring in the future.
- Incorporating additional privacy reviews as part of our design and development processes: Security and privacy have always been front of mind at Mixpanel, but we’re adding some additional explicit checkpoints in our product development processes to help ensure that we’ve considered all of the impacts of the changes we make.
- In-depth security/privacy audits of key existing product areas: We’ve learned a lot from this issue, and our team has been diving in to look for similar cases where these same kinds of problems could arise.
- Operationalizing our response tooling: We’ve built new tools in response to this issue to help us identify the scope of data collection, limit access to data, and to purge it from our systems quickly. We’re taking these tools and making them more general purpose so that we can respond more quickly in the unlikely event that a similar problem occurs in the future.
- Data filtering and detection: We’re exploring capabilities that can detect something like this sooner including changes to the SDK to give us more insight into what data is being sent to us, integration with Data Loss Prevention (DLP) solutions, and even using our machine learning capabilities to detect anomalous ingestion.
We are continuing to conduct a thorough investigation of what happened and how we handled it. We believe that we have addressed the ingestion issue with the speed and accuracy required as your trusted partner.
As part of our action plan, we’ve disabled Autotrack by default for all new projects created, and we’ve put the product on hiatus for now. Ultimately, all of our products and features must put the privacy and security of customers’ data first. If we conclude that certain features cannot meet these high standards, even if they provide value to our customers, we will modify or remove them entirely.
If you have questions or for more information, please email “email@example.com”, and we’ll be in touch as soon as possible.