Before Preparing for Pandemics, We Need Better Evidence of Risk

[Full PDF of report is available below]

Perceptions of threat

The world is currently reorienting its health and social priorities to counter a perceived threat of increased pandemic risk. Spearheaded by the World Health Organization (WHO), the World Bank, and the Group of 20 governments (G20), this agenda is based on claims of rapidly increasing infectious disease outbreaks (epidemics), driven largely by an escalating risk of major “spillover” of pathogens from animals (zoonosis). To be globally prepared for such pandemic risk, many quarters have pushed for comprehensive and urgent action, to avert an “existential threat” to humanity.

The G20 has been central to promoting this sense of urgency. As it states in the G20 High Level Independent Panel report ‘A Global Deal for our Pandemic Age:’ 

“without greatly strengthened proactive strategies, global health threats will emerge more often, spread more rapidly, take more lives, disrupt more livelihoods, and impact the world more greatly than before.” 

Moreover, 

“…countering the existential threat of deadly and costly pandemics must be the human security issue of our times. There is every likelihood that the next pandemic will come within a decade…”

In other words, the G20’s report suggests that pandemics will rapidly increase in both frequency and severity unless urgent action is taken.

In response, the international public health community, supported by scientific journals and major media, is now focused on the task of preventing, preparing for, and responding to pandemics and the threat thereof. Over $30 billion annually is being proposed to be spent on this issue, with over $10 billion in new funding – three times WHO’s current annual global budget. 

Reflecting a sense of urgency of living in a “pandemic age,” countries will vote on new binding agreements at the World Health Assembly in May 2024. These include a set of amendments to the International Health Regulations (IHRs) as well as a new Pandemic Agreement (formerly known as The Pandemic Treaty). The aim of these agreements is to increase policy coordination and compliance between Member States, particularly when the WHO declares that a Public Health Emergency of International Concern (PHEIC) represents a pandemic threat.

It is prudent to prepare for public health emergencies and pandemic risk. It is also sensible to assure that these preparations are reflective of the best available evidence concerning pandemic risk, and that any policy response is proportional to that threat. One hallmark of evidence-based policy is that policy decisions should be substantiated by rigorously established objective evidence and not based merely on ideology or common belief. This enables appropriate allocation of resources among competing health and economic priorities. Global health resources are already scarce and stretched; there is little doubt that decisions about pandemic preparedness will have significant implications for global and local economies, health systems, and well-being.

So, what is the evidence on pandemic threat? 

The G20 declarations from 2022 (Indonesia) and 2023 (New Delhi) are based on the findings of its High Level Independent Panel (HLIP), laid out in a 2022 report informed by the World Bank and the WHO, and analysis commissioned from a private data company, Metabiota, and the consulting firm McKinsey & Company. The report summarizes the evidence in two annexes (Figure 1 below), noting in its Overview that:

“Even as we fight this pandemic [Covid-19], we must face the reality of a world at risk of more frequent pandemics.”

while on page 20:

“The last two decades have seen major global outbreaks of infectious diseases every four to five years, including SARS, H1N1, MERS and Covid-19. (See Annex D.)”

“There has been an acceleration of zoonotic spillovers over the last three decades. (See Annex E.)”

By “zoonotic spillovers,” the report refers to the passage of pathogens from animal hosts to the human population. This is the generally accepted origin of HIV/AIDS, the 2003 SARS outbreak, and seasonal influenza. Zoonosis is assumed to be the major source of future pandemics, barring laboratory releases of pathogens modified by humans. The basis of the G20 HLIP report’s sense of urgency is these annexes (D and E) and their underlying data. In other words, it is this evidence base that supports both the urgency of establishing robust global pandemic policies, and the level of investment that these policies should involve.

So, what is the quality of the evidence?

Despite the importance the HLIP report gives to the data in Annex D, there is actually little data to assess. The Annex presents a table of outbreaks and the years they occurred, with no attribution or source provided. While Metabiota and McKinsey are quoted elsewhere as primary sources, the relevant McKinsey report does not include this data, and the data could not be found when conducting searches of publicly-available Metabiota material.

To better understand the implications from the data in Annex D, we created a corresponding “best-fit” table of pathogen outbreaks and year (Figure 1), with official mortality data for the entire outbreak per pathogen (some extend beyond 1 year – see sources in Table 1). 

In order to address an apparent oversight in the Annex D table, we also included the 2018 and 2018-2020 Ebola outbreaks in the Democratic Republic of Congo in our analysis, since there were no large outbreaks of Ebola reported in 2017. This is likely what “Ebola 2017” was intended to denote in the Annex D table. In our analysis (Figure 1) we exclude Covid-19 since its associated mortality remains unclear and its origin (laboratory-modified or natural) is contested, as discussed later.

When comparisons are made between the HLIP outbreaks table and our table of the last two decades, one mortality event dominates – the 2009 Swine Flu outbreak that resulted in an estimated 163,000 deaths. The next highest, the West African Ebola outbreak, resulted in 11,325 deaths. 

Although these absolute numbers are worrisome, in terms of pandemic risk it is necessary to note that the Ebola virus requires direct contact for spread and is confined to Central and West Africa, where outbreaks arise every few years and are dealt with locally. Furthermore, in relative terms, consider that malaria kills over 600,000 children every year, tuberculosis kills 1.3 million people, while seasonal influenza kills between 290,000 and 650,000. So, putting Annex D in context, the West African Ebola outbreak, the largest in history, thus resulted in the equivalent of 4 days of global tuberculosis mortality, while the Swine flu outbreak of 2009 killed less than influenza normally does.

The third largest outbreak listed by the G20 HLIP was the cholera outbreak in 2010, which was confined to Haiti, and thought to have originated from poor sanitation in a United Nations compound. Cholera once caused major outbreaks (peaking between 1852-1859) and was the subject of the first international agreements on pandemics. Improved water and sewage sanitation  has reduced greatly to a point where the Haiti outbreak was unusual, and there has been a consistent overall downward trend since 1859.

In terms of threat, no other outbreak listed by the HLIP over the 2000-2020 period killed over 1,000 people. The HLIP considers this table to show major global outbreaks every 4-5 years, whereas it actually shows mostly small, localized outbreaks of illness dwarfed by the everyday infectious and non-infectious diseases that all countries deal with. There were just 25,629 non-Swine flu and non-Covid-19 deaths over two decades from the outbreaks considered by the HLIP to be severe (it is noted that other outbreaks occurred through this period that the HLIP did not consider sufficiently significant).

Covid-19 has of course intervened – the first outbreak since 1969 to result in greater mortality than seasonal influenza does each year. This mortality has occurred predominantly in the sick elderly, at a median age above 75 years in higher-mortality high-income countries, and in people with significant comorbidities, a contrast to the predominantly childhood deaths from malaria and young to middle-aged adults who die from tuberculosis. Excess mortality rose over baseline but separating out Covid-19 mortality from mortality resulting from the ‘lockdown’ measures, reducing disease screening and management in high-income countries and promoting poverty-related diseases in low-income countries, makes actual burden estimates difficult.

However, if we accept Covid-19 (for sake of argument) as a natural event, then it should obviously be included when determining risk. There are meaningful debates about the accuracy of how deaths were recorded and attributed to Covid-19, yet assuming the WHO is correct in its estimates, then the WHO records 7,010,568 deaths attributed to (or associated with) the SARS-CoV-2 virus over 4 years, with most in the first 2 years (Figure 2). 

Allowing for population increase, this is still higher than the 1.0 to 1.1 million deaths attributed to the influenza outbreaks in 1957-58 and 1968-69, and the largest since the Spanish flu that inflicted a mortality several-fold higher over a century earlier. With an average mortality of 1.7 million per year over 4 years, Covid-19 is not greatly different from tuberculosis (1.3 million), but concentrated in a considerably older age group.

Tuberculosis, however, continues before and will continue after Covid-19, whereas Figure 2 indicates a rapidly waning Covid-19 outbreak. As the first event in 100 years of this magnitude, though little different from major endemic tuberculosis, and against a background that does not demonstrate an overall increase in mortality from outbreak events, it appears to be an outlier rather than evidence of a trend.

Figure 2. Covid-19 mortality, as of January 2024 (Source: WHO). https://data.who.int/dashboards/covid19/deaths?n=c

The second piece of evidence used by the HLIP to substantiate its claim that we are living in a “pandemic age” is research conducted by Metabiota Inc., an independent company whose epidemiology team has since been absorbed by Ginkgo Bioworks. The Metabiota data forms Annex E of the HLIP report (see Figure 3), which shows outbreak frequency of zoonotic non-influenza pathogens over 60 years to 2020, and influenza ‘spillover’ events for 25 years. 

Although Metabiota is cited as the source, the data itself is not further referenced. That said, an identical non-influenza data set appears in an online presentation by Metabiota to the Center for Global Development (CGD) on August 25^th, 2021 (Figure 4). This dataset also appears in a more recent academic article in the British Medical Journal in 2023, co-authored by Metabiota personnel (Meadows et al., 2023). The authors analyzed the Metabiota database of 3,150 outbreaks, including all outbreaks recorded by WHO since 1963 as well as “historically-significant” prior outbreaks (Figure 5). The data used in Meadows et al. (2023) is available in the article’s supplementary information, and former Metabiota staff confirmed to REPPARE that the dataset used in that article, as in the earlier analyses, is now commercially available through Concentric by Ginkgo Bioworks.

The data points are summarized in the HLIP Annex E via two corresponding claims. Firstly, that there is an “exponential” increase in non-influenza outbreak frequency. Secondly, that influenza ‘spillover’ (transfer from animals) has increased from “almost none” in 1995 to around 10 events in 2020. Both claims require examination.

The upper chart in Annex E (Chart 1), if taken to represent the true frequency of outbreaks, does indeed show an exponential increase since 1960. Yet, as Meadows and co-authors confirm in their later paper, this increase in reporting frequency does not take into account the development of new surveillance and diagnostic technologies, which have enabled better (or in some cases any) detection. PCR testing was only invented in 1983 and has steadily become more accessible in laboratories over the last 30 years. Antigen and point-of-care serology tests were only widely available in the past couple of decades, and genetic sequencing only very recently.

Since 1960, we also have had significant improvements in road transport, clinic access, and digital information sharing. As a result, this limitation in the Meadows study raises a key issue. Namely, that advancements in detection technology may account for the large increase in reported outbreaks, since most small and localized outbreaks will have been missed 60 years ago. As just one example, HIV/AIDS was missed for at least 20 years before identification in the 1980s.

What the above suggests is that there are certainly known spillover effects and that these do occur with some frequency and deadly effect. What is less reliable is the claim that there is an increased frequency of zoonosis and/or that the increase in reporting cannot be fully or partly explained by advancements in detection technologies. Determining the former would require further research that could control for this latter variable.

In their presentation to the CGD (Figure 4), Metabiota included the same frequency data above, but also included mortality as a measure of severity. This is important, since it shows an apparent accompanying exponential increase in mortality is solely driven by two recent African Ebola outbreaks. Again, Ebola is a localized disease and normally rapidly contained. If this single disease is removed as a pandemic threat, the data then shows that, after a few outbreaks of less than 1,000 deaths 20 years ago (SARS1, Marburg virus, and Nipah virus), mortality has decreased (Figure 5). The world appears to have become much better at detecting and managing outbreaks (and resultant illness) under current arrangements. The trend in mortality over the 20 years pre-Covid was downward. A prominent study of a larger database published in 2014, by Smith et al., found the same; namely that there was increased reporting of spillover events but with decreasing actual cases (i.e. burden) based on population size.

The second chart in Annex E of the HLIP report, of influenza ‘spillover’ events, is difficult to interpret. Influenza deaths are trending down in the United States (where data is relatively good) over the past few decades. Furthermore, available global estimates are relatively flat, with around 600,000 deaths per year for the last few decades and despite increases in population. 

Thus, Metabiota’s claim of an increase from 1 to 10 spillover events per year from 1995 to 2000 seems unlikely to refer to a real change in seasonal influenza. It is possible that the increase refers to advances in detection. What is more, if only less severe common influenza variants are considered such as highly pathogenic avian influenza (HPAI) types H5 and H7, then mortality has greatly declined over the past century (see graphic from Our World in Data site). The WHO equally notes that mortality from ‘Bird Flu,’ of which we most frequently hear, has been declining (Figure 6).

As the HLIP report’s annexes indicate, the claim of a pre-Covid increase in outbreak risk appears unfounded. This is good news from a global health viewpoint but raises concerns in relation to current G20 recommendations, since they aim to invest considerable new resources in pandemic policies while potentially diverting from existing programs.

Unfortunately, the McKinsey & Company report cited by HLIP sheds no further light regarding risk. With its focus on financing, the McKinsey report merely recommends an investment of $15 to $25 billion for two years, then $3 to $6 billion annually, summarizing the justification for this investment as:

“Zoonotic events, in which infectious diseases make the jump from an animal to a human, touched off some of the most dangerous recent epidemics, including of Covid-19, Ebola, MERS, and SARS.”

Yet, the evidence for this claim is weak. As shown above, Ebola, MERS, and SARS caused less than 20,000 global deaths between them in the past 20 years. This is the mortality rate of tuberculosis every 5 days. While Covid-19 had much higher mortality in terms of relative disease burden, it is not the “most dangerous” health threat by a considerable margin. In addition, separating the risks from SARS-CoV-2 virus from the risks resulting from policy responses is tricky, and research in this area remains scant. Yet, understanding this separation of Covid-19 risk would be crucial for determining what is or is not “most dangerous” about an outbreak as well as what resources and policies would be best placed to protect us from these future dangers.

Elsewhere, publications on pandemic risk have claimed more than 3 million deaths per year. These numbers are achieved by including the Spanish flu, which occurred before the advent of modern antibiotics and mainly killed through secondary bacterial infections, and by including HIV/AIDs, a many-decade event, as an outbreak. Both influenza and HIV/AIDS already have well-established international mechanisms for surveillance and management (although there could be improvements). As shown above, influenza mortality has been dropping with no outbreaks above seasonal background for 50 years. The type of setting in which HIV/AIDS arose and was able to transmit widely unrecognized for decades can no longer be found.

So, is there an existential risk?

An existential threat is understood as something that would cause human extinction or would drastically and permanently restrain humanity’s potential for survival. In this regard, when we think of an existential threat, we generally think of a calamitous event such as a planet-altering asteroid or thermonuclear warfare. Although we agree that it is reckless to argue that there is no pandemic risk, we also believe that the evidence base to support the claim of an existential pandemic threat remains largely underwhelming. 

As our analysis shows, the data upon which the G20 has justified pandemic risk is weak. The assumptions of a rapidly increasing threat drawn from that data, which is used to then justify huge investments in pandemic preparedness and a substantial reordering of international public health, are not based on solid ground. Moreover, the likely impact of the surveillance structures being put in place to detect natural threats must also be questioned, as the savings claimed are predominantly based on historic influenza and HIV/AIDS, for which mechanisms are already in place and risks are reducing, while mortality from spillover events from animal reservoirs, the basis of the G20’s claims of increasing risk, is also low.

Covid-19 alone also presents a poor justification on various levels. If it is of natural origin, then based on the G20 data it could be understood as an isolated event and not part of a trend. In addition, Covid-19 mortality is predominantly in the elderly and already ill, and complicated by changing definitions of attributable mortality (of, as versus with, the pathogen). If SARS-CoV-2 is laboratory-modified, as some have argued, then the massive effort underway to build surveillance for naturally occurring threats would not be justified nor appropriate to the task.

As a result, we must ask ourselves whether this is an adequate justification for rushing new international legal agreements that could divert significant resources from larger disease burdens that pose everyday risks. The G20 is basing its recommendation for over $31 billion a year in new pandemic financing on mortality figures that pale beside the everyday health risks that most humans face. In effect the G20 is asking countries with endemic infectious disease burdens orders of magnitude higher than these small outbreaks to divert limited resources to intermittent risks largely perceived to be threats by richer governments.

As we have argued, major shifts in policy and financing should be based on evidence. This is currently difficult within the international public health community, as much funding and career opportunities are now tied to the growing pandemic preparedness agenda. Moreover, there is a general feeling within global health policy circles that it is essential to capitalize on a “post-Covid moment” without delay, since attention on pandemics is high and opportunities for policy agreement more likely. 

However, to maintain credibility, the onus is to provide rational and credible evidence of the risk of outbreaks in the context of overall health risks and burdens. This is not reflected in the G20’s statements, indicating that the advice on which they base their claims is either poor, rushed, and/or being ignored. 

There should be time, and urgency, to fix this evidence gap. Not because the next pandemic is just around the corner, but because the costs of getting things wrong will have long-term implications that may be far harder to address once wholesale changes are initiated. As a result, what is prudent is to give the evidence pause for thought, to identify the knowledge gaps, address them, and to pursue better evidence-based policy. 

a Assumed to refer to the 2016-2017 0utbreak. Mortality not recorded, but derived here from attributable child mortality based on Brazil data (0.1203 Zika, 0.0105 background, 0.1098 attributable, in 3308 Zika-positive pregnancies, derived from Paixao et al. (2022); https://www.nejm.org/doi/pdf/10.1056/NEJMoa2101195

b HLIP Report may have intended to reference 2018 (f).

c Attributable chikungunya mortality is normally minimal, predominantly associated with mortality in the sick elderly. WebArchive includes a now-deleted PAHO report including 194 Caribbean deaths in two small island States, which may be an attribution error. https://web.archive.org/web/20220202150633/https://www.paho.org/hq/dmdocuments/2015/2015-may-15-cha-CHIKV-casos-acumulados.pdf

d Median of range derived from the WHO.

e Avian influenza has low mortality throughout the 20-year period – see Figure 6.

f Includes two outbreaks in that year; 45 in India and 8 in Bangladesh.

g Two 2018 Ebola outbreaks added to table, as this may have been what HLIP intended when referring to a 2017 outbreak.