Vince Maniago, VP of Product Management, Personal Capital
We can now group our user experiences, trend them, and figure out, ‘Generally, are we doing better? Are we doing worse? Where can we improve?’ With Mixpanel, our decisions are backed by data instead of conjecture, meaning we can move forward with real product management work.
Founded in 2009, Personal Capital is an online financial advisor and personal wealth management company. Headquartered in Redwood Shores, California, with offices in San Francisco, Denver, Dallas and Atlanta, Personal Capital serves more than two million people with its free planning and analytics tools. It also manages more than $12 billion in assets for 22,000 investment clients in all 50 US states (as of Dec 31, 2019). Any reference to the advisory services refers to Personal Capital Advisors Corporation, a subsidiary of Personal Capital Corporation. Personal Capital Advisors Corporation is a registered investment advisor with the Securities and Exchange Commission (“SEC”). SEC registration does not imply a certain level of skill or training.
Personal Capital’s success depends upon its ability to offer a stress-free, secure, and engaging suite of products that give users a 360-degree overview of their finances, offer solutions, and provide access to registered financial advisors for qualified customers. Maintaining a high level of customer satisfaction is crucial: would you trust a company that offered a less than impeccable experience for users of their technology? Because providing superior customer experience to users of their platform is business-critical for Personal Capital, measuring that experience is a central component of product management and development.
Vince Maniago is the VP of Product Management at Personal Capital. “When I joined Personal Capital five years ago, we were relying on Google Analytics. Unfortunately, our access to real-time data was poor and it wasn’t the best fit for what our team needed. A/B test results were often delivered on the following day, or sometimes, the day after,” said Maniago. “I often had to turn to an engineer or data analyst to pull the data that I needed, but that report could be dated by 24-72 hours.”
As the company continued to build out their product team and strengthen their engineering organization, they started to look at where they could make improvements, particularly their onboarding process. “At that point, we knew having real-time access to data was crucial. If we wanted to take bigger and bigger swings at improving our metrics, we needed to ensure we hadn’t made any major missteps. We couldn’t afford the lower conversion rate if one of those experiences tanked the overall onboarding funnel,” said Maniago.
Personal Capital required the ability to see data by the hour—or even the minute—and that’s where Mixpanel came in. Maniago had previously launched the platform at two other companies and decided to bring it into Personal Capital. He noted, “Prior to Mixpanel, the process of accessing our data was like going to a doctor’s office and receiving my blood test results in the mail. With Mixpanel, my doctor is texting me the results as soon as I walk out of the office.”
“Data represents truth. Mixpanel helps us gain real-time access to the truth, enabling us to make faster decisions. We always want to drive more iterations on the loop: ideate, test, measure, iterate, and do it all over again. The faster we can close the loop, the better we can build our products. On top of that, we’re a startup, so we always have to be mindful of our runway. With Mixpanel, we can now iterate in a day or two, whereas previously it could have taken a week,” said Maniago.
Although timeliness of answers was important, Maniago said there was an even more crucial challenge that Mixpanel helped solve. “Mixpanel has helped us democratize data across the company, especially within the product management organization,” he said.
Previously, the company was using clunky Google Analytics reports or reports generated by their data team. Adding a field or slicing the data a different way meant a wait of days or weeks to get a new version of a report. Once the product team and lead engineers were up to speed on Mixpanel, they were able to instrument new events quickly and to slice and dice data using the reporting interface. Mixpanel helped the Personal Capital team build strong discipline around both making data-informed decisions when designing a test or feature and monitoring how a particular feature performs over time.
“Product analytics are the foundation for how we make all of our decisions. Whether we’re conducting an A/B test or developing a new experience, Mixpanel helps us understand how customers and prospects are interacting with our product,” said Maniago. “We can now group our user experiences, trend them, and figure out, ‘Generally, are we doing better? Are we doing worse? Where can we improve?’ With Mixpanel, our decisions are backed by data instead of conjecture, meaning we can move forward with real product management work.”
The Personal Capital team wanted to use Mixpanel to help them:
A key part of the onboarding funnel is entering your credentials and linking external financial accounts (401ks, mortgages, loans, credit cards, checking, savings, etc.) so you can get a comprehensive view of your assets via Personal Capital’s dashboards. Linking your first financial institution unlocks the magic of Personal Capital; without it, users will not understand the value of seeing all their accounts in one place.
“With Mixpanel, we could see what percentage of people added their first financial institutions within 24 hours and wanted to increase that conversion rate. We ran several multivariate experiments and discovered that displaying logos of various popular financial institutions so that a user could easily choose one and click on it resulted in much higher conversions compared to the other option where users had to search for their financial institution,” said Maniago.
The team also experimented with the number of logos (e.g. 8 logos, 12 logos, or 16 logos) and which logos were shown. “These were among the first experiments we ran which led to an outcome that everyone at the company could get excited about. They now had a tangible understanding of the impact of our testing; it went way beyond experimenting with shades of blue,” said Maniago.
Mixpanel enabled the team to turn around a new experiment in the onboarding funnel about once every 10 days (a limit imposed primarily by volume). Previous decisions might have taken months to make—or might not have been data-backed at all. As Maniago pointed out, “The ability to run two-week iterations that yielded concrete, proven results was a major improvement.”
Personal Capital also offers users access to registered financial advisors, and getting online users to schedule an appointment with an advisor is a key success metric for the company. The team could see in Mixpanel that 40% of people who searched for an available time to meet with their advisor did not actually end up setting an appointment.
Maniago and the team dug more deeply into the problem. The scheduling interface was very straightforward, so how did users end up changing their minds over the course of a few clicks? Perhaps the drop off was attributable to the schedules of advisors, who have limited availability, other clients, and fixed hours? Were users not able to find a time that worked for them? Were they not sure what their appointment was going to be about, or if their advisor could even help them?
To close that understanding gap with real-time user research, Maniago turned to Mixpanel’s messaging tool. For anyone that fell out of the funnel, they would receive an email two minutes later asking what their preferred meeting time was. After reviewing responses, the team was able to form hypotheses around why these users weren’t setting appointments. As a result, the team ended up developing an internal feature where if someone went through this process and didn’t set an appointment, Mixpanel would trigger a webhook notifying Personal Capital’s sales team, who would then call them right away to figure out a time that worked for their schedule.
This kind of data-informed decision-making helped Personal Capital increase the conversion rate of the scheduling experience, and provided valuable feedback into the product development and design process so that they could design an appointment scheduler that would convert better than the previous versions did.
Personal Capital’s marketing team is constantly experimenting with different landing pages to see how various messages drive registrations. For example, they could have one landing page with a message around retirement and another landing page with a message around seeing all your assets in one place for budgeting purposes.
“It’s easy to determine which of these experiences drives the highest registration rate, however, as with a lot of modern online services, getting someone to register is only half the battle. Do our users continue to engage? Do they complete the onboarding process? Do they bail and delete their accounts?” said Maniago.
He continued, “We’ve instrumented in these key elements of participation in Mixpanel, which means that for each landing page, we can see how many people complete actions that correlate with being longer-term users, like filling out the questionnaire, linking an investment account, etc. This information helps our product and marketing teams understand which campaigns are, in fact, working the best to drive users that will continue using Personal Capital for the long run.”
“The online support within Mixpanel is amazing; I almost always end up with someone who goes above and beyond to resolve my issue—usually in real-time. It’s like having an extra person on our team. If it’s during the Mixpanel’s support team’s hours, I can share reports with them and they can walk me through anything I have questions with. They’re always courteous, fast, and explain things well.
This caliber of support has been great for me as an individual user of Mixpanel, but it’s also even better as the company’s Mixpanel champion. I’ve onboarded two to three dozen people at Personal Capital onto Mixpanel and, given the tremendous support experience, I know I won’t have people coming up to me every five minutes to ask a question—they can just reach out directly to Mixpanel for help.”
“Implementing Mixpanel was a smooth process for us and the documentation is very helpful. Once my engineers are up to speed, they do the right thing 95% of the time without even being asked.
Mixpanel is now part of how we build our product. It’s like your favorite pizza place that has great semolina flour from Italy—it’s something they use for every single pizza they make and it raises the overall quality of all their products. Mixpanel has done the same thing for us; it’s one of our key ingredients.”
“One of things I love the most about Mixpanel is that it’s easy to learn. If a product manager has never used Mixpanel before, I can give them a 5-10 minute tour and show them how to use Funnels and Insights, dig into User Profiles and Live View, and they’re off to the races, building funnels and writing reports. In fact, every PM has their own individual Funnel reports to manage their features and areas of responsibility.”
“As product managers and designers, we love Mixpanel’s User Profiles feature. For example, we can look at an individual’s profile and see, for example, that they logged in from the iPhone app at 9:02 am, received an interstitial during that login experience, took action on that interstitial, spent two minutes selecting a time to meet with an advisor, and went on to the next step. Being able to observe, step by step, how a user moves toward conversion means we can plan and design the most effective flows for our users—it’s almost like doing user research at scale.”
“I highly recommend the book, Lean Analytics: Use Data to Build a Better Startup Faster (by Alistair Kroll and Benjamin Yoskovitz). It’s a great resource to help you understand engagement metrics like virality, growth, retention and referral funnels, as well as getting better at developing empathy for your users.”
“It’s easy to get excited about tracking data in Mixpanel, but taking a step back and investing some time in planning upfront pays huge dividends on the other side. The value that you get out of any product analytics tool is dependent on the quality of your implementation—there are no shortcuts here. Spend an hour thinking about which KPIs are going to be really important when measuring the success of your product, which will make deciding which events to track that much easier. Mixpanel also provides best practices guides that will help you understand how to structure your data and events to ensure they scale going forward.”
“As your product starts to mature, you may develop a steady state expectation that a certain funnel will yield between 20-30% of conversions per day. Custom Alerts allows us to insert some bumpers around our conversion funnel so we receive email notifications if anything starts to change. You don’t have to hit refresh on your Funnels all day to see what’s going on.”
“Think about how you can include your product analytics tool in the day-to-day and week-to-week operating of your business. It will make your team’s analytics and experimentation muscles that much stronger.”
Vince Maniago, VP of Product Management, Personal Capital
Mixpanel is now part of how we build our product. It’s like your favorite pizza place that has great semolina flour from Italy—it’s something they use for every single pizza they make and it raises the overall quality of all their products. Mixpanel has done the same thing for us; it’s one of our key ingredients.