Blinded by a Blizzard of Numbers: A Review of Spiegelhalter and Masters

Walk into any bookstore during the holidays of 2021. Observe the goods peddled to those trying to make sense of the pandemic exist in abundance. Most are pretty dire, putting the events in larger historical or geopolitical perspectives that we love to do these days (see the number of publishers slapping “and the future of capitalism” on their authors’ titles, or the many published works that portends to explain “the modern world”).

David Spiegelhalter, a long-term statistician and educator at Cambridge University, whose best-seller The Art of Statistics: How to Learn from Data has taught many how to think better about numbers, decided to join the chorus. Together with Anthony Masters of the Royal Statistical Society, he has just released Covid by the Numbers: Making Sense of the Pandemic with Data. During the pandemic, the duo has written plenty for the left-wing British newspaper The Guardian, and not always particularly well. 

Their short-form book is a pleasure to read: no fluff, very brief chapters, plenty of graphs and much nuanced discussions of what that data might mean. They openly admit that chronicling the statistics of what recently happened while the event still unfolds might not be the best idea; no matter how fast the publishing process, by the time the book hit the shelves, many of their numbers were outdated, and some of their conclusions even undermined. 

The goal isn’t primarily to describe what happened, but to analyze the meaning of numbers that we have been inundated with over the last two years. Readers can easily skip ahead to the questions that most interest them. It’s intended, not as a critique of policy decisions, but to “draw some statistical lessons from this last year.” 

True to Spiegelhalter’s ethos as a statistics educator, the authors state, “We have written this book because we believe that better attention to statistical issues could have improved understanding.” They focus entirely on Britain, and only sparsely compare its experiences with other countries. 

One oddity is that they cite almost exclusively government websites or sources which, in an event where political competence, megalomania, and regulatory capture is deeply called into question, is very suspect. 

The duo carefully explains the many issues in testing, how cycle thresholds, false positives, and testing regimes might distort the interpretation of case numbers and positive results. Britain didn’t suffer from a “case-demic” and another oft-repeated scare, that of hospital capacity, doesn’t hold much water either. They illustrate how as large numbers of non-Covid treatments (surgeries, cancer screenings, minor injuries) were cancelled, resources freed up for other areas of the hospital. Like so many stories from Sweden, Italy or New York City, many of the express hospital facilities erected were superfluous: 

“Seven new Nightingale hospitals were rapidly put together, but hardly used, partly because referring hospitals were unable to spare accompanying staff. The 4,000-bed facility at London’s ExCel centre reportedly treated 54 patients in the first wave. The total cost of these now-closed hospitals was over £500 million.”

The authors are crystal clear that the age-distribution of this disease makes it very specific, where risks for the elderly are one hundred or one thousand times what they are to younger people. Refreshingly, they repeat the same analysis for the cost-benefit assessment of vaccines and their side effects; for younger age groups, their discussion suggests that the risk-reward tradeoff from the vaccines might not be worth it. 

One segment is illustrative of how infected and disingenuous Covid debates have become, a two-sides-of-same-coin problem that Tim Harford, another British statistician, has been so good at capturing. Because many Covid statistics have room for interpretation, there is a lot of material for dishonest interests to either maximize how bad deaths are or minimize them. 

In Chapter 15, the authors give us a brief comparison to other historical harms: the pandemic saw the largest increase in Britain’s crude mortality rate since the Blitz during World War II. That sounds awful, and emphasizes the shocking and horrific nature of the pandemic. If we age-adjust the mortality rate for an older population, Britain in 2020 faced a setback of about a decade. Both those statistics are true; emphasizing one lets you tell the one-sided story you want. 

One oddity is their discussion of risk factors (Chapter 13), and how non-White Brits faced worse risks of death, but adjusting for location, economic deprivation, and preexisting factors mostly equalized the death rates. They conclude that “increased risks were not genetic, but associated with living circumstances and factors such as occupation and access to health care” (had anybody argued otherwise?!). 

What’s so strange about the section is the effort to debunk what seems like a complete strawman is wholly disproportionate to the many other risk factors they report. Age, naturally stands out, but for one word, the authors overlook obesity, which in their graphs presents a bigger hazard ratio than any differences between ethnicities. Where is the analysis about obesity? Where is the implication (and suggestion) that one can take at least some responsibility for one’s own virus protection by eating or living better?

On the same topic, the elephant in the room is Vitamin D, a discussion that is almost completely absent. The authors describe the protective effects of Vitamin D supplementation as “unknown,” and cite a Harvard Health site that dismissively states that “There is no evidence that taking high-dose vitamin D protects you against getting infected.” (This is followed by a confused comment about the quality of observational studies and an admission that Vitamin D deficiency seems to be a risk factor). 

Yet, Vitamin D deficiency has looked like a risk factor since the Spring of 2020; one can argue that the jury might still be out, or the effects of (certain) supplementation might not be adequate, or the effects specifically for SARS-CoV-2 aren’t clear, but “unknown” is misleading. It’s well-established that Vitamin D is involved in many protective functions in your body, and that many people are deficient during the winter months. The brief rejection is both overdone and uncalled for. 

Ivermectin receives a similar treatment, and the authors reveal their allegiances by simply stating that “regulatory authorities recommend against its use,” referencing the FDA. There’s something deeply unsatisfying about accomplished statisticians, with lots of data and studies at their disposal, merely invoking appeals to political authority and moving on. In a recent Guardian piece, the authors do describe that the evidence for Ivermectin has deteriorated as of late, in part from a pulled pre-print and some poorly done studies. 

…And Vaccines

The authors spend a fair portion of the book on the vaccines, and don’t have too much to show for it. Beyond describing a half-dozen vaccines and some of their clinical trial results, and the risk-reward analysis already mentioned, we don’t learn very much. 

At one point they even belittle the risks by comparing adverse effects to other small and insignificant risks that people happily engage in – skydiving, surgeries with anesthesia, or the much worse contraceptive pill! On statins, a drug taken by millions to reduce cholesterol levels, Spiegelhalter and Masters write:

“In contrast to a one-off vaccination, statins are taken daily, and there is the option of stopping or changing the prescription. On the other hand, statins only help the recipient, while vaccinated people can help others through reduced transmission.” (emphasis added)

Giving the authors the benefit of the doubt – a good 7 months have transpired since they completed their manuscript – both these points have been seriously undermined by later developments. Vaccination doesn’t seem to prevent much transmission, and it’s now clear that Covid vaccines are not a one-off, but a recurring Pharma-as-a-service intervention.  

Ironically, statins have for years been subject to precisely the same criticism that many of the Covid-19 vaccines now face: that their relatively minor benefits to some targeted groups are not worth the harms done to the millions of people for whom they’re prescribed. 

There is much to dislike in Spiegelhalter and Masters’ book on the plague year, but considering the partisan and authoritarian nonsense, garbage advice, and terrible statistical blunders we’ve grown used to, the book comes across as fairly balanced. They have some clear blind spots (vaccines, effectiveness of lockdowns, Vitamin D) but there are much worse things to read than Covid by Numbers.