“The reality is that media businesses have lived off the incredible inefficiency of the advertising market for their entire lives. Now it’s becoming efficient, and that makes life harder.”
In fewer than 50 words, Nick Rockwell has just described the existential crisis of the news industry. And if there’s anyone qualified to summarize this, it might be him. For the past year, he’s served as the Chief Technology Officer of The New York Times. He lives this tension daily: New media versus traditional media; what the public needs versus what the public wants; user interests vs. advertiser interests.
“There are these incredible platforms like Facebook that just take the data-driven advertising opportunity to its extreme in a way that can’t be competed with,” he says, sounding more awestruck at the implications than afraid.
The Times is the strongest news brand in America, but even the paper of record has been forced to acknowledge the impact of platforms like Facebook. While the Times may have a different value prop than a social network, they go head-to-head for clients, vying for readers’ attention and advertisers’ dollars.
In that sense, something like a social network is a very real competitor. And to do battle with the bright-faced boy kings of Silicon Valley, the Gray Lady was in need of a touch-up.
The first problem Nick recognized coming aboard the Times was how the engineering organization was siloed from the rest of the company.
“It was really negative because it actually undermined the idea of engineering being sort of a core capability of a company like the Times,” he says. “One of the things I addressed immediately was breaking those barriers down and collaborating as deeply as we could with the rest of the company.”
While more of a cultural quandary than a technical one, this is essentially the role Nick has assigned himself: To make engineering a core capability of the paper’s editorial mission, not just an after-the-fact update.
Technology can improve the overall experience of the Times, he believes, without having to colonize it.
“Traditionally, newspapers couldn’t get any information on what was working,” Nick says. “You knew if the circulation was going up or down, but you actually had no feedback in terms of what people were actually reading, what they appreciated, or why they were remaining a customer or not.”
Some papers cling to bygone business models, while others bend to appease the trends du jour. Circulation, for example, is an outdated metric for a very tech-savvy organization. But newer players overindex on clicks.
“Focusing on clicks would be a terrible mistake for us,” Nick says. “We know from the data that the cycle for a reader to grow to understand the Times and eventually become a subscriber is measured over many years. Sometimes as long as a decade. The signal that you get from just looking at clicks is not going to be significant enough.”
Luckily, there’s a lot more data than just clicks. The Times’ website and apps ingest multitudes of different behaviors. A newspaper is a portal to the outside world. How readers choose to engage with the outside world incidentally reveals a lot about themselves.
“People who consume multiple sections are more likely to become loyal readers,” he offers. “We know that and we work that into our Audience Development strategy across the board. We’re constantly trying to understand where those levers are.”
Because the profile of a Times reader can be complicated, sometimes built over several years, Nick isn’t ready to productize everything he has data on yet. In the scope of the paper’s 165-year history, his team is relatively new. They still need to know how to successfully engage their readership and stay relevant.
“We haven’t landed on what we think the perfect formula is,” Nick says. “Amongst the things we care about are what people click on, the time they spend with a particular piece of content, scroll depth, as well as repeat visits over a defined period of time. Those are some of the key metrics.”
The problem here is that tech platforms like Facebook and BuzzFeed have already locked on their perfect formula, reaped users, and then sold those off to advertisers. So while there can be no question of the Times’ value (one need only count the Pulitzers—117, by the way), it will have adapt to the new realities of content.
All of which Nick knows, but it’s not so easy.
“Personalization is an area where you can expect to see a lot more from us in the future,” Nick says. “However, we need to be careful because the editorial perspective of the Times is one of the things that our subscribers value. They expect us to curate things they need to know, not just the things that they want to know. We’re thinking about how can we create a level of abstraction around the editorial judgement that we supply, so it can be potentially worked into an algorithm.”
As Facebook has proven recently, some editorial supervision is required when developing news algorithms. To that end, Nick says, “If the value for a core subscriber is breadth, you wouldn’t necessarily want to insulate them in their own personal feed of the things that they normally read.”
And so making the broadsheets less broad is still a big what if. While the big innovations of content distribution are on Nick’s mind, his team wants to develop methodologies that, though high-tech, remain true to the Times.
“We’re going to take our time to do this right,” he says. “The test cycles are long because our behavior cycles are long. This work will take place over the next 3-to-5 years.”
Understanding that the Times’ behavior cycles are long like a newspaper’s (and not short like any other app’s) is no small thing. It means that the engineering organization has to align its definition of success with the more traditional notions of the editorial pool.
“We have to allow data into our decision-making process, and then we need to really work to understand the long-term patterns and correlations as we try to build loyalty in our readers.”
And then there are those advertisers. They used to be at the mercy of the papers, but the power balance has shifted. As the Times tries to stay beloved to readers, it also needs to prove efficient in the eyes of advertisers.
“In any media business, you have the interest of the marketers and the interests of the audience, and they’re not completely aligned,” Nick says.
A clear example of this would be ad speed. A high-speed ad is ostensibly good for advertisers. But an ad that loads too fast threatens to turn off readers. And then the company’s reputation suffers.
“There’s a great deal of value to us from that relatively small number of highly-engaged users, which isn’t true of most media sites,” Nick says. “To put it into context, it’s common for subscribers to turn over hundreds of pages in a month. A typical media site, you would be happy to get 10 or 12 pages views a month from a unique user. We might get 200. Sometimes more.”
With the Times, visitors remember. Every experiment Nick’s team runs will have to account for this affinity. When the engineering organization tampers with the interface, it’s not just good reporting that’s at stake. It’s a brand that lives in people’s hearts.
“We really, really can’t alienate those people, or our business is going to suffer,” Nick says. “If you’re dealing with people who come to your site once a month, it’s easier to throw a bunch of ads on the page and hope for the best because they’ll probably still click on the link on Facebook next time and won’t even remember your site.”
The Times’ uniquely deep engagement is what makes unobtrusive advertising important. But it’s also the thing that attracts advertisers in the first place, and Nick’s mindful of that. He wants to do right by both sides.
“If the overall experience loads quickly and performs well, but the ads take several seconds to load in, that’s a problem, but it’s a problem that can be balanced within the interests of our advertisers,” Nick says. “If it’s loading slowly, ads are probably going to get less engagement. But maybe it’s worth it if the reason they load slowly is to capture more data.”
Ad speed and site performance is a problem many media sites are contending with. The more ads, the more revenue. But if the ads’ presence tarnishes the reading experience, readers might drift to competitors. Recognizing this, Nick wanted to quantitatively measure the cost and benefit of these tradeoffs. But over time, gauging how readers felt about this proved nearly impossible.
“It’s pretty easy to measure the technical performance of this—how long the overall various events get fired by the document model, when pages get done loading—but this doesn’t really tell you much,” Nick says. “There’s a whole bunch of scenarios in which it doesn’t give you any information about things that are aversive to the user experience.”
The team knew this tension existed, but articulating how users were feeling it was impossible to do quantitatively. So they had to rethink how they measured.
“There is an idea,” Nick says. “One which I haven’t seen too much of in the industry.”
How does the Times feel?
“The idea we’ve been experimenting with is doing this qualitatively,” Nick says. “We have to test what the impact is for our audience, which is a surprisingly difficult thing to do. We want to capture what the perception and sentiment are about the performance of the overall experience. Then, if we’re able to, I want to isolate the impact of ad performance in particular, which will be really difficult.”
In engineering organizations, qualitative analysis is sometimes looked down upon. But on Nick’s team, there’s a feeling that it may be the most accurate method.
So, to minimize the risk of aberration, Nick’s team is being very shrewd about the types of queries they use. Briefly, he considered doing a comparative approach (How do you feel about the Times’ ad experience vis-à-vis The Wall Street Journal?) but later decided against it. Now, he’s trying to baseline what the experience of reading the Times is like.
“Through surveying and focus groups over a period of time, we’ll follow whether users feel like our advertising is getting less intrusive or more intrusive, which will allow us to steer the ship down the optimal path,” he says. “My instinct is to focus more on generic assessment of the overall feeling of the performance and the pleasure of the user experience, and use those as our guide.”
It’s a slower process than alternative types of analysis, but if it stands a chance of delivering big value advertisers while protecting the brand’s integrity, it’s worth it. Asking users how the Times makes them feel might sound squishy for an engineering organization, but this is a business with a heart, one that’s been ticking for almost two centuries. There can be exactitude in this science, but empathy has a place, too.
“What I see is an organization that really, really wants to change, but which has deep-seated instincts that are different from what you would see in a data-driven organization,” Nick says. “The industry is in a tough spot, but I feel very optimistic for us.”