How the United Nations built (and measured) its data marketplace
“When I was living in Ethiopia and Eritrea, there were times where I’d be in my tent in the middle of the night inputting data into Excel. Or sometimes I would have to iron old maps flat in order to digitize them,” says CJ Hendrix, Information Management Officer at the UN Office for the Coordination of Humanitarian Affairs (OCHA).
At that time, CJ was a Geographic Information Systems Analyst. In other words, he was a cartographer doing mapping and data management during humanitarian emergencies. But after being in the field for several years, CJ joined the OCHA team in Geneva: “I bring the on-the-ground perspective to the organization’s data management,” he tells me.
In 2014, CJ and a small but dedicated team created the solution to the data problems analysts often solved by hand. Collecting, cleaning, and standardizing data is foundational to coordinating relief efforts around the globe. But to transform the way organizations access and contribute data to the humanitarian sector was an enormous undertaking. Instead of reinventing the entire system, people like CJ needed a data marketplace so there was a faster exchange of information.
The answer was the Humanitarian Data Exchange (HDX), an open-source platform dedicated to humanitarian data. This central repository was built so that organizations could contribute data sets while also getting the intel they need when deploying resources to the right people in the right places during a crisis.
With HDX, the United Nations launched the next generation of data-driven humanitarian aid. And despite the fact this product served the public sector, it functioned just like technology in the private sector. Similarly, HDX’s open-source platform needed the analytics to measure how effective they were at accomplishing their original mission – to help improve lives with data.
And as it turned out, the metrics that mattered most to this data marketplace came down to usage – from the data supply-side and the data demand-side. And after looking to Mixpanel for user analytics, HDX found a unique ratio that helped them understand how well they were serving their community.
Getting sand out of the gears
Used by people in over 200 countries and territories around the world, HDX has become the platform the UN, NGOs, governments and humanitarian actors can depend on when coordinating data-driven relief efforts. In fact, the United Nations (along with the New York Times and The Economist) relied on HDX as the common platform for data during the Ebola Crisis.
“There’s a lot of sand in the gears of humanitarian operations, and a lot of that sand is the fragmentation of data and lack of standards,” CJ says. “The data people need isn’t often readily available. Rather, it’s on someone’s laptop somewhere.”
The United Nations has always had a rich history of collecting information (“There are documents and documents and documents,” CJ remembers). But until recently, information didn’t often come in the digital forms a data scientist or analyst could easily use. The most critical data was usually housed on paper or PDFs.
“HDX isn’t trying to add some new functionality to the operations of humanitarian coordination,” CJ continues. “Rather, we want to ‘open-source’ the data and get the sand out of the gears, so there’s a faster flow of information. During a crisis, data used in the right place at the right time can improve lives through quicker and more efficient aid.”
Measuring HDX’s impact, in this case, was going to require unique KPIs. And there would be two sets of metrics depending on whether an analyst contributes to the platform or pulls data out of it. But before HDX could get an accurate picture of the activity on their platform, they needed an analytics framework to understand their users.
Measuring the data marketplace
“Originally we were using Google Analytics, but it became too dense for our entire team to navigate,” CJ says. “So much of what we measure on our platform is event driven, so when we learned about Mixpanel, we liked that it was based on tracking events as opposed to measuring page views.”
Considering most of HDX’s “content” (e.g. data sets) comes from other organizations, the team needed to be able to report back to its contributors on how their data sets were being used.
“Our platform is becoming more visible to the international community and our contributors are organizations like United Nations High Commissioner for Refugees (UNHCR), World Food Programme (WFP), and the US State Department,” explains CJ.
“Often contributors (and their managers) want to know if they should continue to invest time and resources by putting data sets up on HDX. So being able to answer their questions via user analytics satisfies their curiosities and allows us to give them the metrics they need.”
In the near future, HDX aims to build out a new user interface to include dashboards connected to Mixpanel’s API. That way, contributors will be able to see real-time updates on the data sets they’ve added.
Sharing the data sets’ success metrics actually perpetuates a positive feedback loop on the platform. Users see the number of downloads or shares their data sets get, and the positive reinforcement encourages their organizations to continue to add to the platform.
In HDX’s case, similar to other tech platforms, impact can be measured by how well a product serves its users or empowers them to do something. And with an analytics framework, organizations can measure qualitative questions like “making an impact” with specific quantitative KPIs.
HDX has monthly team meetings that center around analytics. Often, understanding engagement and usage drive the questions they care about.
“We measure the the top datasets and ask why these are the best performing,” CJ says. “Sometimes it’s not obvious why some are outperforming others. Downloads aren’t directly proportional to world events. Because of that, we have to hypothesize why one data set would be downloaded, as opposed to others.”
By diving into user behavior via Segmentation or discovering the breakdown of users’ workflows via Funnels, HDX is able to iterate and improve their platform just like any other open-source product. But by focusing the team on the data, HDX answers more strategic questions about who relies on HDX, where and why.
It’s with this user data that CJ and the team are able to measure if they are serving their target audience, and thus their overall mission.
The ratio that matters
When it comes to the demand for data in the humanitarian sector, not much has changed since CJ’s days doing fieldwork. If anything, the demand for data has only increased.
“It’s really important that a majority of our users on the platform come from the field offices,” says CJ. “Of course, HQ offices need data to support relief efforts remotely. However, we like to see an uptick in data set uploads and downloads from field offices because people on the ground are closest to the issues they are trying to solve.”
Because of this unique North Star metric, C.J. instrumented HDX’s analytics so they are able to measure the platform’s usage by field officers compared to those agents working from HQ offices. The HDX team monitors this ratio to see that it’s still on the upward trend.
As of now about 30 percent of those using the platform are from locations around the world where OCHA has field offices. And similar to any tech product, the goal for HDX is to see usage amongst field offices to regional offices go “up and to the right.”
Internal awareness is the best way for HDX to see this uptick. With increased communications like webinars, HDX is building awareness amongst teams under the OCHA umbrella and with other humanitarian actors.
What’s next for HDX
Keeping their mission in mind – to make data easy to find, share and use for analysis – HDX is focused on expanding their scope. By the end of 2017, the organization is launching the Centre for Humanitarian Data based in The Hague, The Netherlands.
The Centre is part of the UN’s Agenda for Humanity, which aims to improve the way organizations coordinate within the humanitarian sector.
“In order to manifest our new vision – to create a future where everyone has access to the data in order to make responsible decisions – we’re now tackling larger initiatives, such as data literacy and data policy,” says CJ.
Throughout his analytics career, CJ Hendrix has seen the humanitarian sector take large strides when it comes to accessing, cleaning, and standardizing data. And as HDX continues to grow, humanitarian relief efforts will only become more data driven.