LinkedIn’s AI makes small talk for you

LinkedIn Smart Replies is an artificial intelligence product with a clear purpose. You get a message, and it provides three possible  responses. The replies themselves tend to be pretty straightforward – “Sounds good,” “Works for me,” and “Sorry, I can’t.” It’s not exactly Her, but it wasn’t intended to be.

LinkedIn Messenger’s goal is not to use AI to prove a theory. They’re not trying to beat the best humans at board games, then beat the robots that beat the best humans. “Our goal is to help users have smarter, more productive conversations,” Arpit Dhariwal, a Senior Product Manager at LinkedIn who works on Messaging, tells us. At LinkedIn, the end product is the end product; they want to build something that people will use.

If the company’s goal is to connect the world’s professionals, and the Messaging team’s goal is to help users have smarter, more productive conversations, their implementation of AI had to accomplish those things. So Arpit had to run some tests.

With any product, the basics are the same: know your user and solve their problem.

In the case of LinkedIn Messaging, Arpit’s team was not looking to build a general-interest machine learning messaging product or just “do machine learning” because it is trendy right now. LinkedIn Messaging is solving for a different set of problems than those that face products like Facebook Messenger or Gmail. It’s a professional audience of people who are perhaps unlikely to have much of a pre-existing relationship with their interlocutors. So first and foremost, he wanted people to feel like they’re actually having a conversation.

“We didn’t want members to feel that they’re talking in an empty room,” Arpit says. “A key component of an active community is having the feeling of being heard. And a big insight we had is that professionals are busy—okay, it’s a little obvious, but it guided us as we built this product. We had a problem to try and solve: How can we give time back to busy people? LinkedIn’s mission is to connect the world’s professionals. Having that clear mission gives us an easy sense of which products to build, and which to throw on the scrap heap. Even just thinking in terms of machine learning, there are literally thousands of different ways you could go, and products you could build. But every company has finite resources, and being mission-driven helps you make the right investments.”

The narrow band of communications that occur on LinkedIn Messaging are perfectly set up to use the value AI in a specific, useful way. The conversations on that product are professional, not personal, so the range of conversational topics is tighter. There is no need to worry about the kinds of in-jokes and more informal, familiar conversations that users might have on other platforms. But that doesn’t mean there aren’t challenges. With any new product, companies need to worry about irritating their customers, but particularly with AI. Nobody wants to make Clippy 2.0.

“There were two main things we thought about with this. First, we wanted to keep the phrases short and simple. Things like ‘Sounds good,’ or What time?’ are simple enough that it’d be impossible to say if a human or a machine wrote them. Second, we wanted the UI to be unobtrusive. If the user wants to use ignore Smart Replies, or build a longer message, using Smart Replies as a jumping off point, they can do that.”

But designing a machine learning algorithm that will reply to users in a succinct, professional way takes resources. And in order to take resources from LinkedIn’s Relevance team, where they employ natural language processing scientists off of other tasks, Arpit needed proof of concept. He needed to show that there was something to this idea.

So they built a minimum viable product, put it on the market and saw what happened next.

Rather than invest the resources to build out a full AI product from the jump, Arpit’s team started small. They built a simple, static quick reply product that gave users three reply options: “Thanks,” “Not sure” and a thumbs up emoji. Not exactly the whole range of human experience, but enough to get started. If users liked “quick replies,” they’d definitely like Smart Replies. If they were not interested, Arpit could kill the product, and move on to another experiment. In short, users were interested.

“We saw a really good lift to some of our key metrics like messages sent, response rates, and at that point in time, we realized that we’re on to something with the static replies. If we started adding intelligence to this thing, it could literally be magic,” Arpit tells us.

With the initial hypothesis that users would respond to quick replies validated, Arpit’s team set their sights on making Smart Replies. He gave a problem statement to the natural language processing scientists designed around two concepts: speed and volume. “We view this particular product’s potential as increasing the speed with which users are able to reply, and since they’ll have to spend less time thinking up and writing out their reply, hopefully they’ll be using the messaging product more often.”

Once they had the resources allocated to this project, it was time to eat their own dogfood. “Some of the initial models were questionable. LinkedIn is a professional network, and some of the initial models’ responses were not up to that standard. Once we dealt with those issues, we rolled it out to a few product managers, and engineers. But we started getting the best insights once we rolled it out company-wide to all 10,000 LinkedIn employees. That really helped us shape the product because as part of messaging team the way I use the product is very different than even someone who’s part of the sales team using the product.”

“It was exciting,” Arpit tells us, “to roll a product out to one percent, five percent, and uniformly see that lift in our key metrics, and know that we had something we could roll out to our whole customer base and know it would help them have better conversations.”

“We didn’t view the Smart Replies as an experiment, or a risk of any kind, really,” Arpit tells us. “The experiment was the static quick replies. Once we saw that product getting lift, we knew if we invested more resources into building a better product, we’d get even more lift.”

“It’s a lot less stressful building a product you know has a market.”