Running smarter A/B tests carefully

Data privacy laws are evolving constantly to better protect consumers’ personal information. The GDPR in Europe, the CCPA in California, the LGPD in Brazil—these laws give more data transparency and agency to consumers, while limiting certain data collection practices. 

And while increased data privacy is a good thing for all of us, these laws have implications for your experimentation practice and tech stack. Many technologies were designed before these laws existed and, while some are iterating quickly to be compliant, others are falling behind. 

So, how do you evaluate your A/B testing toolkit to make sure you’re collecting the information you need to deliver delightful digital experiences while protecting customer data? That’s what we’ll discuss in this post.

First things first: Why is data privacy so important?

More and more people are going online to accomplish everyday goals—things like banking, grocery shopping, and connecting with friends and family. And they’re creating tons of data with every click and query. Companies use this data to create more relevant products, advertising, and shopping experiences. But more data also means more risk for data breaches and irresponsible data handling.

Due to recent high-profile data breaches and exploitative business practices, governments and consumers alike have started to demand more from companies that are collecting and storing their data. The calls have been for more transparency and control over how collected data is being used and more consequences for companies that handle data irresponsibly. This spurred the introduction of laws and regulations around the world, all designed to prioritize consumer data protection.

With new rules and coverage of data breaches, consumers are paying more attention than ever before to how companies handle their data. According to McKinsey, six in ten consumers in Europe now realize that rules regulate the use of their data within their own countries, up from four in ten in 2015. 

But this increased awareness of rules has not meant an increase in overall trust. In fact, consumer trust in how companies are handling their data remains low (under 25% of consumers) across most industries, McKinsey’s data shows.

This means there’s an opportunity for companies that do prioritize data privacy. And a great place to start is by auditing your digital tech stack to make sure the tools you’re using to test and optimize your digital experiences are also prioritizing data privacy.

Before we get into our checklist to help you audit your tech stack, let’s take a quick look at what you need to know about today’s data privacy laws.

What you need to know about today’s data privacy laws

The last several years have seen a bunch of new regulations and requirements around data privacy, including: GDPR and ePrivacy Directive in Europe, CCPA in California, LGPD in Brazil and more. There’s already a lot of great content that dives into the nitty gritty of each of these, but here’s the TL;DR:

These regulations exist to give individuals more control over their personal data. Among other things, companies need to be able to:

• Clearly inform individuals about what’s being done with their personal data,
• Obtain explicit consent to use an individual’s data,
• Provide a lawful basis for using that data,
• Allow users to retract consent at any time,
• Respond to and act on requests from individuals regarding their personal data.

The ePrivacy Directive, informally known as the “EU Cookie Law,”  is of particular importance to conversion optimization practitioners because it’s focused on cookies, which are widely used in marketing and product technologies like analytics and A/B testing tools. 

What are cookies?

Cookies are generally understood as small text files placed on a site visitor’s computer, which are then read by a browser to determine and enable a host of services to that site visitor. But EU law takes a broader approach and as outlined in Article 5(3) of the ePrivacy Directive, cookies are anything that accesses a site visitor’s or app user’s local device memory.

This expanded definition of cookies means any technology that accesses a site visitor’s computer memory infringes on the privacy rights of that computer’s user. Keep in mind, the directive doesn’t prohibit the use of cookies. Site owners or publishers can use cookies if the site visitor provides consent.

How to audit your A/B testing practice and tech stack for data privacy compliance

Experimentation is a critical practice for every marketing and product team. Consumers want data privacy, but they also want increased personalization and more delightful, usable experiences. Testing allows you to iterate constantly based on statistically significant insights about what users want from your site or product.

But digital experimentation requires data collection. From developing an experiment hypothesis, to delivering a more personalized experience, to tracking the efficacy of a test—the practice is powered by data. Which means experimentation teams must make data privacy a top priority.

In that vein, here’s a helpful checklist to consider to make sure your experimentation practice and the technologies that power it are privacy compliant.

• Understand where your technology vendor stores its data
• Understand how much user data the technology vendor needs to collect to perform its function
• Observe if user data is exposed in irresponsible ways, such as the tech vendor’s demo environment
• Understand how the technology is implemented on your website or in your product environment

Remember, a data privacy and security compliance infrastructure is only as strong as its weakest link, so each of these facets carries the utmost importance.

Mixpanel + AB Tasty: A match made in data privacy heaven

Mixpanel enables teams to easily and securely create complex events and leverage the data in AB Tasty for experimentation, personalization and feature management throughout the customer lifecycle. The integration enables AB Tasty to transmit campaign, test, and variation data to the Mixpanel platform, allowing teams to analyze the user interactions and in-product behavior. Teams can make informed decisions triggered by user behavior, and measure that impact on key product metrics.

As the leading provider of experimentation, personalization, and feature management tools, AB Tasty is committed to user data privacy. An EU-based technology company, AB Tasty complies with the strictest privacy laws and regulations, globally, to ensure consumer rights and consent are honored to the highest degree. Together, AB Tasty and Mixpanel can help you run smarter A/B tests and personalization campaigns, all while ensuring the strictest of privacy standards are met, and that security standards can scale for the long-term.