Metric partnerships: global burden of disease estimates within the World Bank, the World Health Organisation and the Institute for Health Metrics and Evaluation

Introduction

At the 71^st World Health Assembly in Geneva in May 2018, a political alliance was struck between Chris Murray, Director of the Institute for Health Metrics and Evaluation (IHME), and Tedros Adhanom Ghebreyesus, Director-General of the World Health Organisation (WHO). IHME and WHO signed a Memorandum of Understanding (MOU), attempting to resolve what Lancet editor-in-chief Richard Horton called “a Cold War” of methodological, institutional, and philosophical tensions that occasionally erupted between the two organisations since IHME was established in 2007¹. Since IHME started putting out the results of its Global Burden of Disease (GBD) study in 2012^2,3, there have been controversial differences between IHME’s estimates of disease burden and those of WHO^4,5, particularly around malaria⁶, tuberculosis⁷, causes of child death^8,9, and maternal mortality¹⁰.

The explicit agreement by IHME and WHO to produce a single GBD study—“a series of capstone papers summarizing high-level findings would be published in The Lancet¹¹” before being used in official WHO documents—raises important questions about what an alliance of global burden of disease estimates means for global health governance and health policy in the global South. In official language, these estimates’ producers have always meant them to go hand in hand with the development of better data collection and vital registration systems in countries with limited data collection infrastructure^12,13. In the 2018 MOU itself, in fact, both WHO and IHME assert that “estimates are no replacement for data from strong surveillance systems¹¹”. In reality, producing burden of disease estimates has always required WHO to reconcile the differences between its own disease estimates and country reported numbers. With this new alliance, the organisation will also have to reconcile estimates and methodologies from IHME with its own estimates and country reported numbers, as well as the two very different philosophical perspectives of the two institutions. These acts of reconciliation are mirrored by the attempted alliance between the World Bank and IHME to produce the Human Capital Index and the methodological tensions revealed between the two organisations with regards to the use of global disease burden estimates¹⁴.

The practice of estimation in health development is by no means new. Early examples of the use of mathematical models based on assumptions include Daniel Bernoulli’s 1766 predictions of smallpox morbidity and mortality rates, should the English government not take on inoculation practices^15,16. However, as has been remarked by many scholars¹⁷, the amount of money and labour that went into estimation-production radically increased in the era of the Millennium Development Goals and is expected to grow in the era of measuring the progress of the much more complicated Sustainable Development Goals. This includes IHME’s production of the SDG-index, which was created particularly with the aim to assess the SDGs’ measurability¹⁸.

In this article, we present first a summary of the origins and evolution of the GBD, which has been the object of much scrutiny¹⁹, over the past 25 years. Then, we analyse two illustrative examples of GBD estimates and the ways in which they gloss over the assumptions and knowledge gaps in their production, highlighting the importance of the historical context by country and by disease in the quality of health data. We show how these estimates gloss over their assumptions and knowledge gaps, attempting to quantify context and data quality. Finally, we delve into the question of the end users of these estimates. Who uses these estimates and who does not, and what tensions lie at the heart of producing estimates of local, national, and global burdens of disease? These tensions bring to light the different institutional ethics and motivations of IHME, WHO, and the World Bank, and draw our attention to how the production of data and estimates is key to representing problems and their solutions in global health. With the rise in investment and power of global health estimates, the question of representing global health problems becomes even more entangled in decisions made about how to adjust reported numbers and statistical equations. The tensions at the heart of the GBD study are of utmost importance because those who represent global health problems are also those who determine how money and influence flow to address them.

Methods

This analysis paper relies on two data sources. First, we used published articles to construct a timeline of the history and methodological conflicts of the global burden of disease study. Second, we analysed popular media representations of partnerships between WHO, IHME, and the World Bank, as well as field notes taken by M.T. at three public events at IHME and the World Bank. These three events were IHME’s 20th Global Burden of Disease Anniversary event in September 2017, IHME’s Annual Board Meeting in June 2018, and Sir George Alleyne’s lecture on human capital at the World Bank headquarters in July 2018. The paper does not attempt to give a comprehensive history of the GBD study—which over the course of its twenty-five-year history has included a very large cast of contributors, including those who have generously provided feedback on the first version of this article, and which has been the topic of extensive debate—but instead provides an introductory history of the development of the study and key moments in this debate in order to discuss the implications of two recent attempted and achieved partnerships between IHME, the World Bank and WHO (the Human Capital Project with the World Bank, and the 2018 MOU with WHO to create a single set of global health estimates).

Key historical moments and tensions at the heart of the Global Burden of Disease study

The GBD and the disability-adjusted life year (DALY) have been subject to detailed scrutiny since their inception in the early 1990s. The DALY and the GBD first came onto the world scene out of a partnership between Chris Murray, Alan Lopez, and Dean Jamison^a, with the explicit institutional support of Harvard University, WHO, and the World Bank²⁰. The Bank requested “a comparative, comprehensive, and detailed study of health loss worldwide to provide the basis for objective assessments about the probable benefits of applying packages of interventions²¹,” as part of the Bank’s increasing influence and financial investment in international health development in the 1980s and early 1990s²². The team’s work culminated in the publication of the 1993 World Development Report (WDR), Investing in Health²³, which synthesized decades of work in health economics into the new health metric, the DALY, to provide punchy, useable language for justifying public and private, national and international investment in health. Murray and Lopez themselves define the study as a “systematic scientific effort to quantify the comparative magnitude of health loss from diseases, injuries, and risks by age, sex, and population over time²¹”.

The three explicitly stated aims of the GBD were “(i) to decouple epidemiological assessment of the magnitude of health problems from advocacy by interest groups of particular health policies or interventions; (ii) to include in international health policy debates information on non-fatal health outcomes along with information on mortality; and (iii) to undertake the quantification of health problems in units that can also be used in economic appraisal²⁴”. The study’s architects were motivated by providing estimates on morbidity rates, as well as mortality rates, on diseases and conditions from a neutral, objective perspective. These estimates were built to be plugged into economic appraisal calculations to give international health organisations and governments guidance for prioritising health interventions based on economic reasoning. Until that point, assessments of global disease morbidity and mortality rates had been the responsibility of different programmes at WHO that focused only on specific diseases and interested largely with mortality rates, with estimates that leant heavily toward overestimation²⁵. The DALY, as a metric which quantified morbidity over time, was introduced as a means of drawing global attention to diseases and injuries that burdened populations but did not always result in death.

For all future GBD studies, 1990 served as the benchmark year, and over the course of the GBD’s life, mortality and morbidity estimates for that year were consistently reworked based on new data inputs and new estimation procedures. The next big moment for the GBD was 1997, when Richard Horton at The Lancet published the first peer-reviewed series of articles based on the research that was at the heart of the 1993 WDR, which Murray and Lopez call the “first complete revision of the GBD 1990 study²¹.” The publication of these four articles^12,26–28 in The Lancet was so foundational to the genesis of the GBD that it served as the starting point for IHME’s 20^th anniversary celebration of the event in Seattle in 2017. Still using the year 1990, these articles addressed a few of the major concerns that scholars had identified in the original study published in 1993, including the criticism of the way the study weighted DALYs by age²⁹. The DALY quantifies healthy time lost to illness or death against a standardised average healthy life expectation, by combining years of life lost to death (YLL) and years of life lost to disability (YLD). The healthy life expectancy (HALE) is estimated for every population represented in the study—determined by location, sex, and year—and it is the reference against which YLLs and YLDs are subtracted to determine DALYs for each population.

In each cycle of the GBD since, its architects have carefully reassessed the nature of the disability weights used to condition DALYs, among other changes over the decades (Table 1). As authors at WHO assert is the case in their description of the evolution of the GBD, determining disability weights has proven to be one of the most contentious issues of producing global health estimates³⁰. Producing disability weights requires an agreement on how to “define, measure and numerically value time lived in non-fatal health states”. At the heart of determining this numerical value are debates about “societal” vs. “individual” judgements on the severity of certain losses of health over others, as well as whether to privilege health professionals’ perspective over the general population’s³¹. In an attempt to address these conflicting perspectives on valuing health, the GBD and GHE since 2010 have based their disability weights on a “comprehensive re-estimation of disability weights” based on surveys from the “general population³²”.

By 2000, Chris Murray had joined Alan Lopez at WHO, and they published their national health systems ranking using the GBD and the DALY, with the explicit institutional support of WHO. They had the particular support of Gro Harlem Brundtland and Julio Frenk, who were, respectively, Director-General of WHO and executive director of the Evidence and Information for Policy cluster at the time. This resulted in the publication of the 2000 World Health Report, Health Systems: Improving Performance. This exercise in explicitly naming and shaming countries for the performance of their health systems was highly controversial, as those countries that scored lower expressed their displeasure to WHO and as researchers and popular media pointed alternatively to WHO’s “underlying pro-market ideology^34,35” and its “Marxist stance^35,36”. However, as public health scholar Martin McKee argued, the 2000 WHR did in fact “place [the] assessment of health system performance firmly on the political and research agendas³⁵”. For the years 1999–2002 and 2004, WHO produced multiple GBD updates and published the estimates largely in the intervening World Health Reports^37–40. These were borne out of the Global Programme on Evidence for Health Policy within WHO, part of the Evidence and Information for Policy cluster of which Murray became executive director when Frenk left to become the Mexican Minister of Health in the summer of 2000.

When Brundtland resigned from WHO in 2003, the GBD lost its most powerful supporter in the organisation, which led to the split between GBD studies, marked by Murray’s departure from WHO for Harvard University in 2003. In 2007, the Bill and Melinda Gates Foundation (BMGF), along with the University of Washington, funded Murray and Lopez to create an independent, scientific institution to produce estimates of global disease burden⁴¹. Part of Murray and Lopez’s vision was to separate the production of estimates from the politics of vertical approaches and disease advocacy at WHO²⁵. IHME’s founding board members included Brundtland, Frenk, and Tedros. Given their large global health portfolio, BMGF also had a clear interest in ensuring timely, independent, and robust production of burden of diseases estimates, and IHME became their guide to do this. IHME lists impartiality as one of its five core principles: “For health evidence to be useful, it also must be credible, generated by a scientific process unimpeded by political, financial, or other types of interference. IHME was created to fill a gap in global health: to separate the measurement and evaluation of health policies and programs from the process of creating, implementing, and advocating for policies and programs⁴²”.

In 2012, the group published their first GBD study in The Lancet, covering the years 1990, 2005, and 2010, calling it the GBD 2010⁴³. As we will touch on more extensively in the next section, this study caused a stir in the global health community, as some crucial estimates, like those of malaria morbidity and maternal mortality, diverged heavily from those put out by WHO and its UN agency partners for the same year⁴⁴. In 2015, IHME published the revamped GBD 2013 in The Lancet, including estimates for all of the years between 1990–2013, and since 2016, they have published complete annual reassessments of these years’ estimates, adding new assessments in each cycle. IHME’s GBD 2017 was published in The Lancet in November 2018.

The DALY, the metric at the heart of the GBD, was introduced explicitly as a means to draw global attention to diseases that do not only kill but also disable. By quantifying the loss of healthy years, the DALY was meant to unveil suffering, like lower back pain or depression, that weighs heavy on the world. Some scholars have argued that the metric, so ready to be used for cost-benefit analysis, and its proliferation more fundamentally have contributed to an intensification of economic rationality in and the neoliberalisation of global health thinking^45,46. Medical anthropologist Vincanne Adams argues that “the DALY provides an economic measure of human productive value by calculating loss of productivity due to disease or disability⁴⁷”. As a tool that was created for the World Bank’s new increased investment in global health development, the DALY has from the start tied economic values to human suffering and attempts to alleviate it⁴⁸. Explicitly meant to be useable for ministers of finance and drawing a particular American version of economics into the sphere of international health development, the DALY has led to the reign of health economics in global health governance, a recent shift which Sridhar calls the economic gaze⁴⁹.

The GBD in its current form is also the result of a particular institutional view on the world. Expanding on Peter Byass’ conceptual framework of global health estimates⁵⁰, we argue that there are actually four different institutional positions that frame global health metrics production. We would call these four institutional positions “traditional academic population health department”, “think tank”, “health-policy oriented multilateral”, and “economic-policy oriented multilateral”, the last three of which are exemplified by IHME, WHO, and the World Bank. We argue that IHME is not a traditional academic population health department governed by academic publishing logics, which Byass contrasts against the governing logics of UN agencies’ global health estimate production. Although there are aspects of academic scientific rigor that define IHME’s estimate production—including oversight by an Independent Advisory Committee—they are also governed by the demand for deliverables by BMGF and have privileged access to private industry’s data streams because of their ambiguous institutional nature. As multilateral organisations, WHO and the World Bank have contractual relationships with officials in the countries with which they work, and thus are responsible for producing statistics that are legible to country officials. With different mandates for the ultimate goal of such production of statistics—one which places wellbeing above all else and the other economic prosperity above all else—WHO and the World Bank nonetheless are both required to harmonise the interests of a wide range of national, international, and civil society actors, including in the process of producing contextualised evidence for disease burden, which IHME asserts is a form of ‘politicizing’ evidence^b. The GBD in its current form has also been born out of collaborations with both WHO and the World Bank, meaning that the current collaboration with WHO is only the latest of a larger history of interactions and compromises between these three frameworks of measuring health’s progress and value in the world.

Conclusion

Fundamentally, the production of the GBD and its centrality to global health governance raises the question of the larger effects of producing numerical assessments of disease burden “from a distance⁷⁵”. How data is collected and how estimates are produced actually shape how we understand global health problems and their potential solutions, and thus estimates should never be taken for granted. When health workers, health policy makers, or even patients do not recognize their experiences in these estimates, like the Malawian physician mentioned above, the usefulness of such estimates and the kinds of knowledge with which they must be accompanied should be assessed. These estimates carry with them particular assumptions about the nature of illness and economics that attempt to universalize experiences of suffering and its impacts on our lives. They are excellent advocacy tools, but unfortunately their power extends beyond merely highlighting a problem, as they also leverage assumed health interventions along with them. What happens far less frequently, with important exceptions^33,76, is the transfer of skills necessary to produce these estimates on a much more local level and investment in vital registration systems and health information systems^77,78.

When numbers are called upon to “speak for” the health needs and priorities of populations in the global South, we must also ask the question of who produces these numbers, their apparent apolitical neutrality, and the broader governance structures that allow for them to hold the power they do. This does not mean that these estimates should not be produced, as global burden of disease estimates are an important “first pass” of the health profiles of different countries. However, they cannot be proxies for suffering in and of themselves, and global health benefits greatly from the visibility of the scientific and methodological conflicts that are at the heart of tensions between the two sets of powerful health estimates produced by WHO and by IHME. Ultimately, more work needs to be done to create evidence that is relevant and meaningful on country and district levels, which means shifting resources and support for quantitative—and qualitative—data production, analysis, and synthesis to countries that are the targeted beneficiaries of such global health estimates.

Data availability

The primary and secondary sources used are cited. M.T.’s field notes from the three public events are under restricted availability. See the American Anthropological Association’s 2003 “Statement on the Confidentiality of Field Notes” for more information. However, highly redacted versions of them may be available for academic research purposes. In order to access these highly redacted field notes, please contact M.T. (marlee.tichenor@ed.ac.uk), outlining why the field notes would be relevant for your research and how they will be used.