There has been little attention paid to Cochrane’s review of Covid‐19 vaccines, in stark contrast to the worldwide media attention paid to its review of face masks.
Since Cochrane reviews have been lauded as “rigorous” and “trustworthy,” it is prudent to apply equal scrutiny to the latest update of the “Efficacy and safety of COVID‐19 vaccines.”
Findings of the Cochrane Review?
Published in December 2022, the review analyses 41 randomised controlled trials of 12 different vaccines, involving more than 400,000 individuals without a previous SARS-CoV-2 infection.
The review indicates that most trials were no longer than two months in duration and conducted before the emergence of variants of concern such as Omicron (up to November 2021).
Compared to placebo (which was mostly saline), the authors conclude with “high certainty” that most of the available vaccines could reduce symptomatic Covid‐19, and in some trials, they could reduce severe or critical disease.
It also concludes that there was “probably little or no difference between most vaccines and placebo for serious adverse events.”
Due to trial exclusions, the authors did acknowledge that the results could not be generalised to pregnant women, people who’d already been infected with SARS-CoV-2, or immunocompromised people.
A Flawed Review?
A critique of the Cochrane review has recently been published by researchers Peter Doshi, Joseph Fraiman, Juan Erviti, Mark Jones and Patrick Whelan.
The critique raises some serious concerns about the conclusions reached by the Cochrane authors.
Doshi et al, are the same researchers who reanalysed the pivotal mRNA trials and found one additional serious adverse advent (SAE) occurred for every 800 people vaccinated with an mRNA vaccine.
This contradicts Cochrane’s review which concludes that there is “little or no difference in SAEs compared to placebo.”
Doshi et al point to significant flaws in the Cochrane review that could explain the contradictory findings.
For example, in Moderna’s trial reports, SAE tables included efficacy data on individuals suffering serious Covid-19, which were almost entirely in the placebo arm. Cochrane did not remove the efficacy data before presenting its analysis of Moderna vaccine SAEs, thus tabulating results that dilute the true rate of harms.
The Cochrane review also shows the absolute difference in SAEs between most Covid vaccines and placebo “was fewer than 5/1000 participants.” That’s equivalent to 1 in every 200 people – which is not rare – and incompatible with Cochrane’s stated conclusion that there is “little or no difference.”
According to Doshi et al, another flaw in Cochrane’s review is the “composite endpoint” used to categorise “severe or critical Covid-19” cases.
Doshi’s team points out that the endpoint includes many participants who were not “severe or critical.”
For example, in the Moderna trial, 21 out of 30 cases were not hospitalised (see table)
There was also a problem with how “Covid-19 cases” were counted in the trials, and this problem carried over into the Cochrane review when the evidence was being synthesised (garbage in = garbage out).
The trial sponsors waited 1 week (Pfizer) or 2 weeks (Moderna) after Dose 2 before counting Covid cases.
Put another way, Covid cases were not counted until 4 or 6 weeks after Dose 1 was administered (see table)
Excluding all cases in the 4-6 weeks after Dose 1 is “particularly concerning” to Doshi et al, because there was very little follow up time before the manufacturers decided to unblind the trials and offer the vaccine to the placebo group (median follow-up 2 months after Dose 2).
Another issue highlighted by Doshi et al, is the fact that the mRNA vaccines were very “reactogenic” and likely to “unblind” the participants and bias the trial outcomes. People in the vaccine group, for example, took more medications to lower their fever following treatment than those who took placebo, making it easier to guess which treatment they’d received.
Nonetheless, the Cochrane reviewers judged the risk of bias related to blinding in the trial as “low.”
The Cochrane review also states that it’s “unclear if and how vaccine protection wanes over time” and that the evidence in their review is “up to date to November 2021.”
However, Doshi et al say that this is incorrect. The Cochrane review includes two papers [1,2] where Pfizer, in fact, did show that efficacy against Covid-19 declines over time – vaccine efficacy had waned to 84% ≥4 months after Dose 2.
Finally, the trials analysed in the Cochrane review mainly included healthy people who’d never been exposed to the virus, which means that the efficacy outcomes are no longer applicable to most people who have recovered from single or multiple infections. In the same way, the trials are not applicable to pregnant women, or immunocompromised people. This was not emphasised by the Cochrane reviewers despite being a major limitation.
Cochrane’s Response?
A media enquiry with a list of questions was sent to lead author of the Cochrane review, Isabelle Boutron, a Professor of Epidemiology at Université Paris Cité and Director of Cochrane France.
Boutron confirmed her team had received the comments by Doshi et al but did not respond to further questions, expect to write in an email:
“We are working on the responses, which will take some time to check and respon[d] adequately to each point. We will, of course, inform you when our responses have been finalised.”
There is no timeframe in which Cochrane authors are required to respond to criticisms of their reviews – some have taken several years. A critique of Cochrane’s review of HPV vaccines submitted by Doshi et al in 2018, still has not received a response.
If Cochrane accepts the criticisms laid out by Doshi et al, and revises its review accordingly, it’s sure to arrive at a very different conclusion than what currently stands.
Republished from the author’s Substack
Published under a Creative Commons Attribution 4.0 International License
For reprints, please set the canonical link back to the original Brownstone Institute Article and Author.
