Ellen Chisa’s uncommon practices for building richer products
After almost two years of building products at Kickstarter, Ellen Chisa started feeling restless. She had left her mark on every part of the product, from relaunching the discovery experience to building mechanisms to help people start projects (and fund them). It was time to find a new gig, get a change of scenery and face a new challenge.
“My friends thought the obvious next step for me was to go hiking in Mexico for three months,” Ellen laughs. “There’s, of course, a time and a place for that. But for me, I feel most creative when I have something to react against, or when I have a system I don’t really like or a system I want to fix.”
Instead of going off on the proverbial Eat-Love-Pray expedition, Ellen opted for an institutional challenge – Harvard Business School. But after studying engineering, working at Microsoft and Kickstarter, she realized HBS wasn’t what she was searching for. It was only in going back to school where Ellen realized that tinkering gave her more joy than being in the classroom. After her first year at HBS, she spent the summer working at Blade, a consumer tech incubator. The project she worked on over the summer turned into Lola, a mobile travel-concierge company started by the co-founder of Kayak, Paul English. After her first day on the job, things crystallized for Ellen: Harvard Business School was far more emotionally taxing than it was inspiring.
“Burnout is not about the number of hours you’re working, but how you feel about those hours,” Ellen says. “I worked 12 hours at my first day at Lola, and I realized they felt less tiring than three hours of class at HBS. I didn’t know if I could go a second year in a row without making things.”
After leaving HBS and returning to her life in tech full-time as VP of Product at Lola, Ellen abandoned the notion that there’s only one way of doing things. How you build a product (and how you measure it) needs to be unique to the users it serves, especially when revitalizing an entire industry like travel service. For Ellen, following generic paths to success wasn’t going to cut it, so she revamped Lola’s product roadmap and dismissed the conventional wisdom around such components like onboarding flows and success metrics. But as Ellen and her team deviated from the normal patterns, a strong analytics framework kept their team honest, steering them in the right direction. Here, she shares what she’s learned along the way.
Charting a new map
Having worked at Microsoft, a huge corporation with established product/market fit, to Kickstarter, a smaller company gaining major traction, Ellen learned multiple ways in which she could organize her product roadmap successfully.
But she knew a product roadmap isn’t just a wish-list of things to be built. It’s also a powerful tool to describe the specific ways the product is growing, where each team’s priorities lie, and how stakeholders are meant to align under one cohesive plan.
Easy, right?
So when Ellen set out to recreate Lola’s product roadmap, she riffed off of traditional frameworks in order to personalize their roadmap to the company mission. In doing so, Ellen radically reoriented the product roadmap around its users.
“Before, I created roadmaps so they were split up between pieces like the iOS section and the technical infrastructure section,” says Ellen. “But in order to encourage people to feel more accountable to all the pieces of the product, I made a shift so that the roadmap was organized based on the end-user.”
Compared to other startups, this is an unusual way to construct a product roadmap, but it worked. “Now, at Lola, the product roadmap is based on whether the product is client-facing, travel consultant-facing, or internal-facing.” With this reorientation, the product organization could stay focused on the audience they were trying to serve.
“By interpreting the product roadmap in a different way, people felt more accountable to all of the pieces they touched rather than just where they had their name on the technology,” Ellen says. “Making that tweak was a small process detail that you wouldn’t really think mattered that much, but it really changed how people thought about the work they were doing.”
Ellen knew that a product roadmap was not only the GPS for the company but also a tool to set the tone for her team’s culture. Like many product leaders who have echoed this sentiment before, the VP of Product’s role is not just about setting up the processes around how decisions will be made. A product leader also has to figure out the right way to motivate and rally teams around a singular vision.
“We’ve been using the same roadmap process for a whole year now,” Ellen says, “I’m happy with its consistency, which is pretty noteworthy considering we’re still an early stage company.”
As Ellen radically shifted the focus of the product roadmap, there were other ways in which Lola’s product team could go against convention, starting with the onboarding process. While many mobile apps urge users to complete the onboarding flow as quickly as possible, Lola made their onboarding process even longer.
A different way to onboard
For those unfamiliar with Lola, it’s a new type of travel company providing on-demand, personal travel service through a smartphone app. But even in this renaissance for travel service, the Lola product team has had to reconcile typical mobile onboarding patterns in order to provide their clients (e.g. users) with the highest level of service.
“When it comes to building an onboarding survey, conventional product wisdom says to ask as few questions as possible,” says Ellen. “However, this was a huge challenge for us because we realized that the more preferences we could get from a user upfront, the better experience we can provide a user.” Other mobile services that depend on stated user preferences, like dating or personal shopping apps, can probably relate to this conundrum.
“There’s a major psychological edge to building an onboarding flow,” Ellen explains. “Especially in the travel sector, getting to know a client’s travel style right away keeps the product from having to stop and ask questions every single time you book a trip. In that way, the onboarding survey helps the travel consultant get acquainted with the client, and we can create a highly personalized experience.”
Following the “best practices” for onboarding – getting a user up and running on an app in as little time as possible – wasn’t actually in their users’ best interest. To be most effective, Lola asks a series of questions to understand who the app is serving. However, the team had to find the right balance in the number of questions and the types of questions they were asking upfront. To strike that balance, Ellen went straight to the data.
“We depended on the data very early on to optimize our onboarding flow,” Ellen says. “We were able to see which questions people were skipping and which ones had the highest conversion rate. If we found that one question never impacted what we booked, we would remove that question from the flow.”
With this granularity, Lola’s product team created the right set of onboarding questions that gave their team the most amount of information, with minimal user friction. And it’s here where the product team also learned that onboarding can be a delightful and fun experience for users. Just like planning a trip can create anticipation, an onboarding flow can build a user’s excitement for using the app, too.
“People love sharing their travel preferences,” Ellen says. “In fact, we’ve even seen how travelers love to commiserate on what they don’t like, too, which is why we ask our users to give Lola their least favorite airlines, hotels, etc.”
In tuning and tweaking this process, Ellen realized there was more to measurement than measuring just onboarding flows. As a new, mobile-first company in a legacy industry, Ellen and her team had to discover what “success” means, for the product and for the greater company mission, especially when there is nothing else to compare to.
Discovering your own perfect KPI
With something as personal and experiential as an on-demand travel service app, Ellen knew that generic KPIs like the number of downloads, flights books, hotels secured, etc. would only be half the story. It’s also about the experience someone has while traveling.
“I personally like the idea of people getting a little bit further outside of their comfort zone,” Ellen says. “If Lola is able to do some of the work on everyday things, like helping you not having to worry about your flight being canceled, maybe that’ll give you the headspace to do something a bit riskier. Maybe it’s meeting up with some locals to hang out in a new neighborhood. Or, better yet, if I hear that a traveler who’s afraid of heights ends up going zip lining, that’s success to me.” The value of traveling with Lola is also measured by how far their customers might’ve deviated from their typical travel patterns and truly embraced the experience.
In order to accurately measure Lola’s unique standard of success – Ellen’s team started combining traditional quantitative measures (e.g. the starred ratings) with the stories and survey results travelers shared by the end of their trip. And just as a product org must design its own KPIs, Ellen also believes, “your analytics should be personalized to the product.”
For Lola, that meant gathering data in a few different ways, like event-based tracking, surveys, and qualitative interviews. But according to Ellen, ultimately, any analytics framework needs to accomplish three things – answer a specific hypothesis, monitor a product’s core functionality, and capture user insights that anyone in your organization could act on.
“I recommend most early stage companies use an off the shelf tool, like Mixpanel,” Ellen says. “Off the shelf tools are good for making data accessible to everyone because most people on your team will never be able to write all their own custom SQL queries – nor should they – so having a solution that’s both sophisticated and easy-to-use is important so people across any organization feel comfortable going to the data.”
Having a solution that gives the flexibility to track and retroactively collect everything will empower other members of the product team to splice-and-dice the data from many angles.
“There’s nothing worse, though, when you think you’re instrumenting something correctly – tracking and measuring all the right data – only to realize you were missing something key from the beginning,” Ellen continues. “But the fact remains, there is no sure-fire route to understanding your users. You have to piece together different trails of information to patch together a cohesive story.”
Despite the popularity of playbooks, frameworks, and “how-to” articles, each business requires its own, unique analytics strategy. But this doesn’t solely apply to just analytics – it’s for all aspects of a product or company. From the roadmap to onboarding and success metrics, every part of a product should be built and measured based off of how it’s serving its users.
And even from her own experience, Ellen knows that generic practices aren’t going to revitalize a stagnating industry. Doing something entirely new will.
Right before our conversation, Ellen pinged me to let me know she was running a few minutes late. She was on “travel agent duty” where she works as one of Lola’s many internal consultants helping users plan, book, and manage their trips.
“I’ve found that the customer journey in the travel industry is more nuanced than just assuming travelers go from planning, to booking, to traveling in one seamless process. For example, the way people book flights is different than the way people book hotels. And it’s different from the way people book rental cars or book cruises,” Ellen says.
As the VP of Product, Ellen sees stepping in as a Lola consultant as necessary field work to truly grasp the nuanced experience of the app. It’s also great fodder for the product org to understand how they can continue to improve the service for its users.
“It’s a little bit weird for me to make a judgment call on how people should travel or what they should do. However, offering people richer travel experiences is an important service, and that’s why Lola is here,” Ellen says. “Because when you travel, and you’re perhaps going outside your comfort zone, you feel more confident when someone is there for you.”
Photos made available through Ellen Chisa and Douglas Fernandes via Attribution 2.0.
