Detected fraudulence aside, there is no stronger criticism of a study than refuting the key result by using the study data. That opportunity does not arise often.
I present a striking example, pertaining to a study from Israel. Trying to be methodical, my article is somewhat on the long side, but the implications at the end are radical and broad.
Goldin et al. estimated the effectiveness of the Pfizer vaccine on several Covid-related outcomes, including death, in residents of long-term care facilities in Israel (mean age of 83 years). The large cohort (over 43,000) was heavily skewed towards vaccinated residents (90 percent). Only about 4,000 residents were not vaccinated.
Using a statistical method called survival analysis, the authors reported two age-adjusted values of vaccine effectiveness (VE) against Covid-related death:
Skipping ten days after the first dose, VE was 72 percent.
Skipping about seven days after the second dose, VE was 85 percent.
Goldin et al. also analyzed all-cause death as an endpoint, which many researchers have omitted. Most important, two of their figures (below) show the cumulative number of Covid deaths and all deaths at several time points — from which we can compute the cumulative number of non-Covid deaths. The latter data has been consistently hidden in studies of vaccine effectiveness.
Moreover, we have mortality data since an “index date,” the date on which the first dose was injected. We can analyze the data the way it should have been analyzed. No skipping.
Source: Goldin et al.
For some reason, the time points for Covid death do not exactly match the time points for all deaths, but they are not too far apart (figures above). Therefore, the number of Covid deaths at the time points for all-cause death (30 days, 60 days, etc.) can be reasonably estimated by interpolation. Then, subtracting the number of Covid deaths from all-cause deaths reveals a crucial piece of data: the number of non-Covid deaths.
My busy tables below show the cumulative number of deaths (Covid, non-Covid) in vaccinated residents and unvaccinated residents by the end of the follow-up (5 months), and by three interim time points. Using simple analysis, formally called “cumulative incidence,” I computed the risk of the two types of death in vaccinated (blue) and unvaccinated (red) residents.
The top table shows that the risk of Covid death was consistently higher in unvaccinated than in vaccinated, but the surprising result is revealed in the bottom table: that was also the case for non-Covid death! The mortality rate from non-Covid causes in 4,114 unvaccinated residents of nursing homes in Israel was 3 to 7 times the mortality rate in their vaccinated counterparts, depending on the follow-up time. Or vice versa — the mortality rate from non-Covid causes was substantially lower in nursing home residents who were vaccinated against Covid. That astonishing result is seen early on, within one month of the first dose.
Does the Pfizer vaccine protect against death from non-Covid causes?
We have yet to hear someone making the claim.
If not, what is the explanation?
It is simple and not astonishing at all. The decision of whom to not vaccinate was not random. It must have been based on reasonable medical considerations, particularly life expectancy. For instance, what is the merit of vaccinating a 90-year-old who suffers from advanced dementia and metastatic cancer?
Those 4,114 unvaccinated residents were sicker to begin with. Their life expectancy was shorter, regardless of possible SARS-CoV-2 infection, and that’s why their non-Covid mortality was several fold higher.
Stated differently, belonging to the unvaccinated group was a general marker of poorer health. Or vice versa — belonging to the vaccinated group was a marker of better health. That’s on average, of course.
The phenomenon we observe here is called the “healthy vaccinee” bias, and it is well-documented in the research literature, dating back to flu vaccines. The bias is very strong in frail elderly residents of nursing homes, but it is seen in all age groups of the general population.
The implication of the “healthy vaccinee” phenomenon — when estimating vaccine effectiveness — is called confounding bias. A naïve comparison of Covid mortality in vaccinated people and unvaccinated people, even if age-adjusted, is grossly misleading because the latter have higher risk of death to begin with. At least part of their higher Covid mortality, if not all, has nothing to do with not being vaccinated. They are simply sicker people.
Even Goldin et al. are aware of the bias, to which they devote one sentence at the very end of the article:
“The unvaccinated group might have suffered from more comorbidities, leading them to be more susceptible to SARS-CoV-2 infection and death, thus making the vaccine’s effectiveness seem higher than it actually is.” [my italics]
Some researchers think that the bias operates in the opposite direction (called confounding-by-indication bias), whereby the unhealthy person is more likely to be vaccinated because they are at risk. Regardless, the net effect of the healthy vaccinee bias and confounding-by-indication bias, if the latter exists, is shown in the bottom table (above): those who were vaccinated had substantially lower non-Covid mortality. They must have been healthier, on average, not the other way around.
The table below shows the risk ratio and VE against Covid death, as computed from the data in the previous top table. VE is around 80 percent when calculated at various time points, and my simplified computation for the entire follow-up (82 percent) is similar to the main result of Goldin et al. (85 percent). Recall that all these estimates are distorted (biased) versions of the truth due to the healthy vaccinee bias (and naively assuming no additional sources of bias.)
Most important, the data on the risk of non-Covid death allow for rudimentary correction of these estimates, which is certainly better than no correction at all. The method is best explained by a simple example.
Suppose the apparent risk of Covid death is twice as high in unvaccinated than in vaccinated, which means a biased risk ratio of 0.5 in favor of the vaccinated, and biased VE of 50 percent. Suppose we find out that the risk of death from non-Covid causes is also twice as high in unvaccinated. What does that imply?
Vaccination has made no difference. It had no effect on Covid death. A two-fold risk of Covid death is the expected, “baseline” risk of death in unvaccinated because they are generally sicker. Vaccinated or not, they would have had a two-fold risk of death from Covid than their vaccinated counterparts — just like their two-fold risk of death from non-Covid causes. The biased risk ratio of 0.5 (VE=50 percent) should be corrected to 1 (VE=0 percent).
To get a risk ratio of 1, from a biased risk ratio of 0.5, we need to multiply 0.5 by 2, which may be called the bias factor. The bias factor captures the underlying higher risk of death in those who were not vaccinated. It may be estimated by the risk ratio of non-Covid mortality, comparing the unvaccinated to their vaccinated counterparts.
In my simple example, the correction method nullified the presumed effect of a vaccine. As we’ll see next, the outcome could be anything from attenuated VE to negative VE, where a supposedly beneficial vaccine is actually harmful.
The table below shows the bias factor in the study of Goldin et al. by follow-up time, along with the corrected risk ratio, and the corrected VE. For example, over the entire follow-up unvaccinated residents of nursing homes in Israel were 3.5 times more likely to die from non-Covid causes than vaccinated residents (bias factor of 3.5). Multiplying the biased risk ratio of 0.18 by 3.5 changed the risk ratio to 0.63 and reduced VE from 82 percent to 37 percent.
Almost all Covid deaths have accumulated by the third month (888 out of 899). Indeed, the biased VE was essentially the same (81 percent). Since the bias factor was higher (4.1), the corrected VE is now 22 percent.
Whether VE was 22 percent or 37 percent — that’s a mediocre vaccine. And worse results are forthcoming.
The biased estimates of VE minimally increased over time (from 78 to 82 percent). The bias factor, however, decreased from 7.3 in the first month of follow-up to 3.5 over the entire follow-up, which is not too difficult to explain. Given the shorter life expectancy of the unvaccinated cohort, the most vulnerable members of that cohort died earlier. The remaining people gradually made up a somewhat “healthier” surviving cohort, thereby narrowing the non-Covid mortality gap between unvaccinated and vaccinated.
By the end of the first month the bias factor was 7.3 and by the end of the second month it was 5.2, whereas the biased risk ratio was similar. As a result, we observe a harmful effect of the Pfizer vaccine in the first month, and overall null effect by the end of the second month. That’s negative and zero VE, respectively, against Covid death.
When the inference strongly depends on the amount of data — no effectiveness by the second month versus 22 percent to 37 percent effectiveness with longer follow-up— we have a rule of thumb: The inference is stronger where we have most of the data, not after adding a few more observations. About 95 percent of all Covid deaths have occurred in the first two months (first row in the table above).
The correction method is not perfect, and the result depends on the value of the bias factor (an estimate by itself). Nonetheless, an increased risk of Covid death during an early post-vaccination hazard period is compatible with other data. Indeed, news outlets in Israel reported outbreaks of Covid infection in nursing homes shortly after the initiation of the vaccination campaign.
Below are two translated paragraphs from a news report, dated January 14, 2021, about three weeks into the campaign:
“Once again, a failure in nursing homes: At the same time as the second dose of COVID-19 vaccines is being distributed, the pandemic is hitting hard in the institutions where the elderly live. In the past two weeks, outbreaks have been recorded in no fewer than 160 geriatric institutions, and 1,098 new confirmed cases have been detected among residents of institutions licensed by the Ministry of Health alone.
In parallel to the surge in the number of patients in nursing homes and assisted living centers, in the past two weeks “Senior Shield” [a task force for Covid management in nursing homes] stopped publishing the daily report on Covid morbidity data in geriatric institutions on the Ministry of Health website.”
Why did they stop reporting? Have they also seen a surge in Covid death of vaccinated residents of nursing homes during the first month of the campaign?
Whether the Pfizer vaccine had time-dependent negative effectiveness, no effectiveness, or mediocre effectiveness— the excellent effectiveness against Covid death, as reported by Goldin et al., was false. Assuming this conclusion is not challenged, what are the implications?
Some readers might think that the refutation of one study does not mean a lot. Goldin et al. are wrong, but there are other studies that support the narrative of a “highly effective vaccine” in the vulnerable population. We did not show that the results from those studies were also false.
That’s not how deductive inference works. If VE against Covid death is shown to be far from “highly effective” in one study of the frail elderly, we must deduce that all other studies that reported similar or better VE are false as well — likewise distorted by the healthy vaccinee bias. Otherwise, we have to make an implausible assumption: Despite severe bias, the play of chance miraculously generated the true VE in the study of Goldin et al.
What should happen next?
First, the paper by Goldin et al. should be retracted.
Second, vaccination of the frail elderly with updated Covid vaccines should be halted.
Third, public health agencies should initiate a Request for Applications (RFA) for placebo-controlled randomized trials of Covid vaccines in nursing homes — with Covid and all-cause mortality as endpoints.
Such trials are scientifically justified because residents of nursing homes, the most vulnerable population, were excluded from the original trials (in which death was not an endpoint). Moreover, randomized trials in this unique population become ethically obligatory when corrected VE against Covid death from observational data ranges from mediocre to negative, and there are vaccine-related fatalities.
Of course, all of the above is relevant and applicable in another universe.
Published under a Creative Commons Attribution 4.0 International License
For reprints, please set the canonical link back to the original Brownstone Institute Article and Author.
