Global Health Metrics Initiatives: Shortfalls, Opportunities, and a New Way Forward

April 16, 2020—Public health relies on numbers. Good data help communities, organizations, and governments make well informed decisions. But numbers only paint part of the picture. In a new Viewpoint published today by The Lancet, Jeremy Shiffman, PhD, and Yusra Shawar, PhD, reflect on the problems and opportunities created by the world’s increasing reliance on global health metrics—numbers used to track global and national trends in wellbeing and burden of disease. Shiffman is a Bloomberg Distinguished Professor of Global Health Policy at the Johns Hopkins Bloomberg School of Public Health and the Johns Hopkins School of Advanced International Studies. Shawar is an expert in global governance of health and an assistant scientist in the Department of International Health at the Bloomberg School.

Shiffman and Shawar acknowledge the value of  global health metrics, such as the Institute for Health Metrics and Evaluation-led Global Burden of Disease (GBD) studies, USAID-supported demographic and health surveys, and the multi-indicator cluster surveys supported by UNICEF. These types of metrics have been crucial in identifying and responding to major changes in the health of populations. GBD studies, for instance, provided valuable data on the rising burden of non-communicable diseases in sub-Saharan Africa. Currently, metrics are a vital tool for addressing the ongoing COVID-19 pandemic.

In their commentary, Shiffman and Shawar give voice to the concerns raised by experts critical of the potentially negative and undue influence these metrics have on public health policy and investment. They also offer approaches to help the global public health community develop metrics that better reflect the lives and priorities of the people these measurements are meant to serve.

Through an extensive review of literature, Shiffman and Shawar synthesize key shortfalls in the production, dissemination and use of global health metrics raised by critics.

Key findings include:

1. Lack of transparency: Estimates are often generated using opaque techniques, producing numbers that are potentially flawed or misleading. Critics also raise concerns that estimates are used inappropriately to evaluate progress on public health and other social welfare goals, such as the Sustainable Development Goals.
2. Adverse effects on national health information systems. Investments in global health metrics may divert resources away from building national data collection capacity. Often, global estimates diverge considerably from national statistics, creating challenges for local policy makers to know how to respond appropriately. Moreover, it can lead local governments to rely excessively on global data that is less relevant for addressing provincial and district-level priorities. 
3. A scientific veneer covering a contingent undertaking. Global metrics are often perceived as authoritative, despite the many subjective decisions about what to measure and assumptions about how to calculate indices. While treated as scientifically valid, many metrics experts and other researchers express concerns about the validity of global metrics. 
4. An oversimplified picture of reality that displaces other valuable forms of knowledge. Numbers have an inherent allure because they’re deceptively simple. But they can’t capture the complex stories that accompany the emergence of suffering and disease.  Social, political and ethnographic research that create a more complete understanding of public health problems take time. While numbers are essential, an exclusively numbers-centric approach to public health can displace other valuable forms of knowledge.
5. Transferring power from local to global organizations.  When powerful institutions fund, produce and promote global metrics, it can crowd out local and national priorities. The imbalance in power can also force local entities to rely on global institutions for data and to set priorities based on global rather than national or district-level concerns.

The authors’ research also led to the formation of recommendations for a new approach to the development, dissemination and use of global health metrics.

They propose three ways forward:

1. Encourage global health metrics and the national health information systems agendas to advance synergistically: Global institutions should acknowledge the potential shortfalls in the data they produce and could design and disseminate their work cautiously and with potential adverse effects in mind. They should also work more closely with national policy makers and researchers to find ways to ensure the two agendas strengthen one another.
2. Encourage pluralism of participation in the production of metrics.  Currently high-income countries and powerful organizations play an outsized role in the development of global health metrics. Including more participation from governments, researchers and public health organizations from low-income countries could help produce data that better serve vulnerable populations.
3. Foster pluralism in the kind of information that matters in global health. Numbers are vital, but so are studies using qualitative methods including ethnography and comparative historical inquiry.  Many excellent studies of these kinds are published in anthropology, development, sociology and political science journals, which receive less attention from the global health community. Inviting these alternative voices to the table can help create a better understanding of the nuances and complexities of human wellbeing.

To read more visit, The Lancet.

Strengthening accountability of the global health metrics enterprise was written by Jeremy Shiffman and Yusra Ribhi Shawar. It was funded in part by the Open Society Foundations.