Do masks reduce risk of COVID19 by 53%? How about 80%?

I came across a headline in the Guardian saying masks reduce the risk of COVID-19 by 53%. The story trumpets a piece published by the British Medical Journal.

Although lofty, this is still a lower number than the estimate offered by the CDC director on Twitter of 80%. Interestingly, in the only published cluster RCT to date during COVID-19, surgical masks had an 11% risk reduction and cloth masks had no effect at all on the primary endpoint of symptom driven lab positive results. That cluster RCT took place in a location with essentially no vaccination (aka circumstances which would give masks the best chance to show the best effect size).

The only other completed RCT during the pandemic, DANMASK was null as to the effect of surgical masks, and had been powered to detect a 50% reduction. At the time, many complained DANMASK was underpowered. Masks worked, but not that well, they argued. Yet, it appears now DANMASK was adequately powered if one is to believe the 53% estimate. So which is it? Was DANMASK adequately powered or not? Is 50% plausible or not?

Before you answer, let’s remember that even the authors of the 53% study write, “Risk of bias across the six studies ranged from moderate to serious or critical.” I never thought I would be wishing for ‘mild’!

With my colleagues Ian Liu and Jonathan Darrow, we performed a comprehensive assessment of all data for community masking during COVID19. It’s a long review— 25,000 words— but I encourage you to read it. Despite what you hear on TikTok, there is no substitute for reading. 

I think it is fairly clear that cloth masking— which remains the predominant recommendation during the pandemic (did any place on earth mandate any other mask?)— has at best weak, inconclusive data and no clear evidence of efficacy. At the same time, given the massive number of mask devotees, I have no doubt that non-randomized studies will find 53, 80, or even 90% efficacy. With enough analyses, we may even get 95%! But that won’t make any of them true.

Here is a thought experiment: Collect a set of non-randomized papers** on the efficacy of masking and a similar set on the efficacy of ivermectin (or hydroxychloroquine or vitamin D). Then ask a group of scientists to score them for scientific validity. I guarantee you: masking will defeat ivermectin. Then swap* the words “masking” and “ivermectin” on all the papers and ask another group of scientists to rate them. What do you think they will say? Then white-out the words “masking” and “ivermectin” and ask a third group to figure out which ones are ivermectin and which are masking. It won’t be pretty.

(*note: you may have to change more than the actual word to disguise it, but you get the idea).

(** note #2: all these studies are unreliable)

Scientists have lost any consistent standards for evidence appraisal. Non randomized data with dirty measures of exposure and unrealistic effect sizes should set off warning bells. Or, if you want to just believe in things, then go ahead, just believe in them, but don’t pretend you are following a consistent framework for evaluating evidence. And no need to publish papers that don’t prove anything or change anyone’s mind.

The truth is we should have run several cluster RCTs in western, high income nations. For kids, adults, in different settings, with variation in masking strategies. We didn’t do it for the same reason people RT the Guardian headline. Faith outpaced evidence when it comes to masks.

COVID19 has already resulted in Instagram censoring the Cochrane collaboration. And now masking has a 53% effect size— well, unless you are talking about DANMASK, which was underpowered, obviously. We might as well give up entirely; throw away evidence based medicine, rip up Sackett’s writings, and let the makers of the Impella tell us how well the product works. We can abolish the FDA while we are at it, and delete clinicaltrials.gov. Preregistration of RCT is a waste of time. Even RCTs are a waste of time. Scientific truth is just what people believe is true, and critical appraisal only applies to claims embraced by the other tribe or members of the other political party. Let’s call it: the new normal.

Reposted from the author’s blog