Listening to customers at Black Ops
By 2015, Matthew Black was a veteran of the startup world. He’d founded two companies; one exited successfully (“a cool ride”), one didn’t (“a colossal failure”). He was ready for a new challenge. His plan, as he recounted to us was “to build a hybrid agency that would leverage technology to build mobile and web experiences for customers.”
He assembled a team, and thus, Black Ops—a certified Mixpanel partner—was born. But it wasn’t until a call from an Australian customer in April 2016 that the agency’s way forward became clear. It was April 2016 and the customer had just spoken with Facebook, who had demonstrated a chatbot. The customer wanted Black Ops to look into it and see if it would make sense to integrate into their products.
“We got on the phone with Facebook, they showed us the conversational bot that they were about to announce at F8. I remember looking over and the whole team’s eyes just lit up. We knew that the time to have a conversational experience with an automated bot was coming, and the time was now.
“So we went heads down, we ended up building a bot for that customer who’d made that call, and for a couple others. It was fun, but frankly, the bots didn’t quite match the expectations of the end user.”
Not long after, Black Ops found itself in an article of 2017 Rock Star Agencies to watch identified as the chatbot agency. “That was news to me,” Matthew told us. “But right after that, we started getting phone calls from some very large customers and saying, ‘We read this article that said you’re the chatbot agency. We don’t know if a chatbot makes sense for us—can we just hire you to figure that out for us?’ It turned out to be a self-fulfilling prophecy: once everyone said we were the chatbot agency, we had no choice but to become the chatbot agency.”
Building the chatbot agency
They had the technical know-how, but they didn’t know how users would react to their chatbots. So they needed to figure out what was working and what wasn’t.
“We built a plugin that sends all the event data to our instance of Mixpanel. That way we can track any message that’s sent by the bot or by the user, what actions the user took, and of course, user profiles. That way we could go back to our customers with a detailed view into how their end users were interacting with the bots. We want to see them going through what we call the ‘conversational funnel’ that runs from saying ‘hello’ to the bot to having their problem solved or making a purchase, depending on the case.”
When it comes to building products, Matthew’s team depends on Mixpanel to inform their product decisions: “We try to install Mixpanel even before we even launch the first product, so we can get our customers familiar with looking at key metrics like usage, retention, and conversion so we agree on what success looks like, and can iterate quickly. That way, it’s really easy to see if one messaging variant is outperforming another, for example, and stick with the one that’s working.”
On selling into the enterprise
Chatbots might seem like a pretty widespread piece of technology, but in the cautious enterprise space sometimes Black Ops gets resistance from companies who only see risk.
“When we’re trying to sell to big companies, it comes down to education and trust,” Matthew tells us. “We never go out to market without a full conversation script present. We literally draft up a document that would probably fill an entire conference room with all the possible scenarios in which a chatbot can respond back to a user so they can see that they’re in a closed system. Marketing departments in particular need to control the the way the company’s message is perceived to the target customer.
“The chatbot needs to control and manage the message. It’s not creating new messages, it’s not changing the message, it’s just scaling the company’s existing message and the messaging around the brand. Once we show the company that the bot can keep the message consistent, that puts them at ease.”
The future of chatbots
For Matthew, leading a company is about two things: vision, and execution. And finding the balance between big dreams and nitty gritty execution can be a challenge.
“I believe that in 24 months or less, if you are having a conversation with a brand, it will be automated. Users will become more familiar with chatbots, and if the chatbot sets the expectations up front for what it can do, most customers are going to prefer an automated 24 hour solution, whether that applies to buying a product, checking out a product, re-ordering a product or learning about new products.
“Now, to get to that future, the technology needs to become more widespread. And that’s our company’s mission. To build a successful company in this space means bridging the present and the future. I love forming the dialogue between integrating humans and intelligent technology. I love doing that and I think anyone who stays focused on that bridge is gonna do very, very well for themselves moving forward.”
You never know who will strike up a conversation with a chatbot
The point of chatbots is to scale communication beyond what is humanly possible. Analyzing their behavior requires a data-informed approach because customers will never behave exactly as expected. There’s no way to know how people will respond, and Matthew has certainly been surprised by user behavior.
“We built a chatbot for a large healthcare insurance company to advertise on Facebook,” Matthew tells us. “They were targeting the 60-year-old and above demographic. And they were skeptical as to whether or not their customers would even be on Facebook. So we did a little research and found that they were, so we built the chatbot and put it live.
They weren’t sure what would happen next. They worried that the older demographic might find the bots off-putting or difficult to interact with and fail to make it through the conversational funnel. “What we found out was fascinating. The average member of the 60-plus demographic who is on Facebook spends up to four hours a day on Facebook, only in two sessions. So this means that they sit down—on their computer almost always—and do a two-hour Facebook session.”
That meant opportunity. Matthew: “When we gave this demographic an experience, and one that they could talk to in a way that replicated the feel of this insurance company’s traditional in-home sales techniques, it was right in line with how they’ve always bought insurance. They didn’t want to call the 800 number because wait times are two hours plus, so we saw them interact with the chatbot in exactly the same way they would with an actual salesperson.”
By responding to real user’s actions, Black Ops can build the best chatbot.
As a certified Mixpanel partner, Black Ops is able to help with strategic support in building out a chatbot strategy and implementation. To learn more, reach out to partners@mixpanel.com.
