Healthy Vaccinee Bias: Letter to the Editor of The Lancet Regional Health – Europe

The Lancet Regional Health – Europe

Dear Editor,

Nordstrӧm et al. have studied the relative effectiveness of the fourth dose of a Covid vaccine against all-cause death, versus three doses, in elderly residents of Sweden. One of their two cohorts comprised 24,524 residents of nursing homes. The authors estimated vaccine effectiveness of about 40 percent in the frail elderly (a rate ratio of about 0.6).

As I will show here, the true effect was somewhere between a rate ratio of 1.2 and 2.4, namely, negative effectiveness. The fourth dose was futile at best, and possibly harmful to this vulnerable population.

The study design was simple. Three-dose recipients were matched to four-dose recipients on several variables, and the matched cohort was followed for death from any cause. The authors skipped the first week after vaccination and estimated the effect in two intervals. The best result (relative effectiveness of 39 percent) was derived from a follow-up period of 7-60 days, in which three-quarters of the deaths have occurred.

The authors were aware of the key threat to the validity of their results: residual confounding by unmeasured health characteristics. They write: 

“Furthermore, although third-dose recipients had similar baseline characteristics as fourth-dose recipients, some third-dose recipients likely did not receive the fourth dose because of deteriorating health that was not captured by the baseline characteristics. If so, this would increase their risk of death and result in a higher estimated VE.”

That’s the “healthy vaccinee” bias, documented repeatedly in various datasets from different countries. Vaccinated are healthier, on average, than unvaccinated, and those who received the N+1 dose were healthier than those who received the N dose. That was true in UK data for third-dose recipients (versus two-dose recipients) and fourth-dose recipients (versus three-dose recipients).

Fortunately, the bias can be removed, at least roughly. Researchers from Hungary and the US (and myself) independently proposed a similar correction method for Covid death, using data on non-Covid deaths. We compute a bias factor – the ratio of non-Covid deaths in the sicker group versus the healthier group – which reflects different baseline characteristics. Then, we multiply the biased risk ratio of Covid death by the bias factor. 

The logic is simple: we adjust the risk of Covid death upward in the healthier group, to create two groups that have comparable baseline mortality risk. The remaining mortality difference, in whatever direction, should estimate the vaccine effect. The correction takes us beyond incomplete adjustment by matching or multivariable regression because it accounts for unmeasured relevant variables.

There is no data on Covid death and non-Covid death in the study of Nordstrӧm et al., but the same correction can be applied to all-cause mortality, the endpoint of the study, as explained next.

That four-dose recipients were healthier is evident in the cumulative mortality graphs, which separate at the very beginning of the follow-up (Figure below). That’s a clear indication of different baseline mortality risk because we don’t expect any benefit of the fourth dose within one week of an injection. Hence, the mortality ratio by the end of the first week should estimate the bias factor, which may be used to correct rate ratios, derived from later, left-truncated, follow-up.

Having no data on the percentage or number of deaths before the seventh day (about 150?), I visually estimated that ratio to be about 4 (enlarged figure on the left). 

To the right of the authors’ table 3 (below), I computed the corrected rate ratio for all-cause death during 7-60 days of follow-up, assuming a bias factor of 4 (visual estimation), 3, and 2 (most conservative). The corrected effect ranges from 2.4 to 1.2 against the fourth dose, a harmful effect.

The same strong bias was evident in a study of nursing home residents in Israel during the first vaccination campaign. After correction, the estimated risk ratio (two doses versus unvaccinated) was 1.6 at 30 days of follow-up and null by 60 days of follow-up. As you know, there are no randomized trials with a mortality endpoint. Bias-prone observational studies are all that we have.

Public health authorities in Sweden and elsewhere recommend another booster to vulnerable, nursing home residents in the coming fall. I am sure we agree that no one wants to recommend an injection that is useless at best, and possibly harmful.

I could have submitted this letter to you, as usually done, and published it here, if rejected. However, I had tried to submit letters three times before and decided to reverse the order this time. Incidentally, my second rejected letter was submitted to the Lancet, and the point I have made there about residual confounding bias was recently exposed (by others) in a letter to the editor of the New England Journal of Medicine.

I hope you will seek the authors’ response, publish this letter in your journal, and consider retraction of the paper by Nordstrӧm et al.

Sincerely,

Eyal Shahar, MD, MPH

Professor Emeritus of Public Health

https://www.u.arizona.edu/~shahar/