The Ins and Outs of Covid Vaccine Safety

At the time of FDA approval for any vaccine, it is impossible to know whether it causes rare, unexpected serious adverse reactions. More than a year after the Covid vaccine approvals, we should have that information, but we do not. This is a serious problem. 

If the vaccines are mostly safe, people need to know that, so they do not hesitate to get vaccinated. If there are serious safety issues, people need to know that, so they can properly weigh the risks and benefits, which vary by age. This failure has forced people to make their decisions based on anecdotal evidence. It has also led to less trust in CDC and the FDA. Unfortunately, this distrust extends beyond the Covid vaccines to other vaccines as well. 

Over the last two decades, I worked closely with the CDC and the FDA to help design the systems used to track vaccine safety after FDA approval. During the pandemic, the FDA and CDC have not used the systems optimally and journalists and the public understand them poorly.

This essay describes the vaccine safety surveillance systems, what they can and cannot accomplish, how they have been used to evaluate the two mRNA Covid vaccines (Pfizer and Moderna), and how they can answer the important vaccine safety questions that we need urgent answers to. 

Pre-Approval Clinical Trials

When the FDA approves a drug or a vaccine, we know its efficacy from randomized clinical trials, but our knowledge about its safety and potential adverse reactions is limited. This is unavoidable. To measure efficacy – whether the vaccine works to prevent undesirable outcomes like infection or hospitalization – it is often enough to evaluate the product on a few thousand people. 

That sample size, however, is insufficient to determine whether the vaccine causes rare but serious adverse reactions. Pfizer evaluated its vaccine on 18,860 people. If an adverse reaction happens in only one in 10,000 people, and we see one or two such adverse reactions in the clinical trial, that is not enough to determine if the vaccine caused the reaction or if it occurred because of chance alone. 

Also, if the randomized trial does not include enough people from important demographic groups, we can say little about its safety in that group. The Pfizer trial did not include many people under 30, above 80 or pregnant women, so we cannot know much about adverse reactions for those groups from the trial alone. 

The pharmaceutical manufacturer actively collects information about adverse events during the trial, and the trials provide the best and most reliable information about common vaccine adverse reactions that occur within a few months after vaccination. 

For the mRNA vaccines, injection-site pain, fever, chills, muscle and joint pain, fatigue, and headache occurred more commonly among the vaccinated than in the placebo group. Because of randomization, we can infer that the Covid vaccine caused these reactions. These mild adverse reactions were expected, as most vaccines cause them, although they are more common than for most other vaccines. 

Post-Approval Vaccine Safety Monitoring

Since clinical trials are too small to tell us whether the vaccine causes rare but serious adverse reactions, it is necessary to do post-market safety surveillance after the FDA has already approved the product. In the United States, the three most important post-market vaccine safety surveillance systems are the Vaccine Adverse Event Reporting System (VAERS), the Vaccine Safety Datalink (VSD), and the Biologics Effectiveness and Safety System (BEST). There are other vaccine safety evaluation systems in other countries. In the United States, we also have the CDC’s After Vaccination Health Checker (vSafe) and the Clinical Immunization Safety Assessment Project (CISA), but they do not have the same ability to evaluate causality as VSD or BEST.

Vaccine Adverse Event Reporting System (VAERS)

Administered jointly by the CDC and the FDA, VAERS is a passive reporting system where anyone can report a plausible or suspected adverse vaccine to the CDC/FDA, including physicians, nurses, patients, families, and friends. Vaccine manufacturers must forward reports that they receive to the VAERS system. Most countries have similar systems not only for vaccines but also for pharmaceutical drugs.

VAERS and other passive reporting systems have strengths and weaknesses but more of the latter. The strength is that it is universal so that an adverse reaction can be reported no matter where or when it occurs. The two main weaknesses are underreporting and overreporting. Overreporting comes from the fact that the vaccine is not necessarily the cause of all adverse events that occur soon after vaccination. That is, many VAERS reports are accidental occurrences unrelated to the vaccine. 

By itself, the number of reported post-vaccination events (strokes, seizures, heart attacks, deaths, etc.) is hence of limited use since those events might have occurred even without the vaccine. The key is whether there are more events than one would expect by chance if the vaccine did not cause them. To accurately determine whether the vaccine was responsible for those events, we need to know precisely how many people were vaccinated, and we need to receive all their health events as well as health events from an unvaccinated comparison group. None of this is available in VAERS.

Sophisticated epidemiological methods, such as ‘proportional reporting ratios’ and ‘gamma-Poisson shrinkage’ can help overcome some, but not all, of these problems. By making raw VAERS counts public without any such accompanying analyses, the CDC and the FDA have generated more confusion than clarity from these data. 

There are two major uses for the VAERS system. One is to find adverse reactions that occur within a few hours after vaccination. That worked for the Covid vaccine – as VAERS quickly discovered a small risk of anaphylaxis right after receiving the Covid vaccine at about one per 100,000 doses. Anaphylaxis is a potentially life-threatening allergic reaction that doctors and nurses can easily treat with epinephrine. 

In late 2020, when the Covid vaccination campaign started, some public health officials proposed drive-through vaccination sites, where people would roll down their car window, get the vaccine, and then drive off. But if anaphylaxis does occur, it is better to have a nurse close by to provide the epinephrine rather than driving on a busy highway. The anaphylaxis finding in VAERS put an end to the drive-through plans. Instead, patients are vaccinated in health facilities and asked to stay around for at least 15 minutes after vaccination.

In their published article on VAERS Covid vaccine data, the CDC provides raw counts of reported adverse events and counts divided by the estimated number of vaccine doses given. Critically, there is no information on whether the adverse events occur more often than one would expect by chance, which is necessary to determine if the vaccines may have caused them. This is not the fault of the highly competent CDC scientists doing the analyses. It is an inherent weakness of the VAERS data. 

The authors of the CDC article wrote that “most reported adverse events were mild and short in duration.”

In a bid to reassure the public about the vaccines, the media used this as a take-home sound bite, but unfortunately, it is nonsensical. Patients care about the likelihood of a serious adverse reaction occurring per vaccination dose; the ratio of mild to serious events observed is irrelevant. A vaccine with one mild and one serious adverse reaction per 1 million doses has a ‘terrifying’ 1:1 ratio. But it is much better than a vaccine with fifty mild and one serious adverse reaction per 100 doses administered, even though the latter has a more ‘reassuring’ 50:1 ratio.  

The second significant utility of VAERS data is to generate a list of potential adverse reactions that researchers can further investigate using the VSD and BEST systems. For instance, after analyzing VAERS data, the CDC authors of the just referenced article concluded that heart disease deaths need to be further investigated to see if the Covid vaccines increase their frequency. Based on the early VAERS data, researchers also identified other potentially serious side effects that needed further investigation, including coagulopathy (inability of blood to clot), strokes, myocarditis (inflammation of the heart), acute myocardial infarction (heart attack), Bell’s Palsy (paralysis of muscles in the face) and Guillain-Barré syndrome (a rare immune system disease).

Vaccine Safety Datalink (VSD)

The Vaccine Safety Datalink is a collaboration between the CDC and several integrated health systems, each of which makes available electronic medical records of patients for data analysis. In VSD, an exposed cohort of vaccinated individuals is defined independently of any subsequent health events. All health care visits are available irrespective of vaccination status, which means that the VSD does not suffer from the same reporting biases as VAERS. 

Researchers can then compare the observed adverse event counts with what would be expected by chance in the absence of vaccination. Researchers estimate the latter using either (i) historical counts in the same population, (ii) concurrent controls of similar unvaccinated individuals, or (iii) self-controls (comparing different time periods from the same vaccinated individuals). Having a control cohort or time period is critically important to determine whether the health events observed in the vaccinated cohort are caused by or unrelated to the vaccine. 

For example, in my own work with the VSD, we learned that the measles-mumps-rubella-varicella (MMRV) vaccine causes febrile seizures in one-year-old children. In the VSD data, there were many more seizures on days 7 to 10 post-vaccination compared to 1 to 6 days or 11 to 42 days post-vaccination. If the seizures were unrelated to the vaccine, we would expect to see approximately the same number of seizures on each day after vaccination. Because of this, pediatricians no longer give the MMRV vaccine to one-year-old children.

The MMRV vaccine is still used for the booster shot, given to 4 to 6-year-old children, for which no such excess risk exists. The toddlers are instead given two separate shots for MMR and varicella, respectively. 

MMRV is a powerful example of the potential of the VSD system, which quickly detected this safety problem soon after the vaccine’s launch. The finding dismayed Merck, the vaccine manufacturer, and others who had promoted the new vaccine. It was a heated conference call, to say the least, when we presented these results to Merck, but the childhood vaccine schedule was altered because of the VSD’s findings.

The VSD has provided definitive proof that Covid mRNA vaccines cause myocarditis. When all age groups were combined, there was no evidence for an increased risk of myocarditis, but there was a strong and clear association for young adults, with young men having the highest risk. The VSD has also confirmed the VAERS finding regarding anaphylaxis. Early analyses of the VSD data did not find any other problems with the mRNA vaccines when all age groups were combined. Neither did VSD find an excess risk of non-Covid related deaths after any of the three Covid vaccines.

Biologics Effectiveness and Safety System (BEST)

Using health insurance claim data, the FDA has built a similar system to the VSD. It got off the ground just before the pandemic, so it does not have as long a track record of experience as the VSD. But the population it analyzes is larger in size, and through the Medicare program, the FDA has better data about older Americans than does the VSD.

In a similar manner to VSD, FDA can track every health care event, including diagnoses, hospitalizations, and procedures, and track vaccinated and control cohorts over time. In July 2021, the FDA reported that for those over 65 taking the Pfizer vaccine, the BEST system had ‘signaled’ four potential adverse reactions: pulmonary embolism, acute myocardial infarction, immune thrombocytopenia, and disseminated intravascular coagulation. The FDA did not provide any data in their announcement, and to my knowledge, they have not published any follow-up analyses. They have provided data on myocarditis.

Vaccine Safety Concerns

Vaccine safety must always be evaluated relative to disease risk and vaccine efficacy. Older adults have a high Covid mortality risk, so unless they already have natural immunity from a prior Covid infection, the benefit of vaccination outweighs the small risk of both known and potentially unknown adverse reactions. Covid mortality is exceptionally low for children and young adults, so for them it is unclear whether the limited benefit of vaccination outweighs the still unknown safety profile of the vaccine. 

We know there is a small risk of myocarditis, but we do not yet know enough about other potential heart problems, nor about the long-term consequences of vaccine-induced myocarditis. A recent CDC study showed a lower risk of myocarditis after vaccination than after Covid infection, but that is not the relevant comparison. Since most vaccinated people will eventually get Covid despite their vaccination, the proper comparison is the risk of myocarditis after Covid infection versus the combined risk of myocarditis after vaccination and after their subsequent post-vaccination Covid infection. 

It is natural that the public has questions and concerns about vaccine adverse reactions, and even more so considering that many governments, corporations, and schools mandate the vaccine. In the United States, the public vaccine safety discussions have primarily focused on pharmaceutical vaccine manufacturers, VAERS data, and anecdotal reports. The pharmaceutical companies do not have the data needed to answer vaccine safety questions properly, and whatever side-effect information reported to them must be forwarded to VAERS. 

While there have been heroic efforts to analyze and interpret the publicly available VAERS data, that is not where the ultimate answers will be found since VAERS cannot establish causation in the way that the VSD and BEST systems can. 

We built the vaccine safety surveillance systems to quickly find any adverse reactions caused by the vaccines when they exist and assure the public about their safety when they are safe. That has only partly happened with the Covid vaccines. Both the VSD and BEST have excellent rank-and-file epidemiologists on staff. The VSD has been able to detect and quantify the increased risk of myocarditis after Covid vaccination and show how that risk varies by age and sex. 

For the mRNA vaccines, the big question that needs an urgent answer is whether they cause an increased risk of heart attack and/or other serious heart problems. There are many anecdotal reports, especially among young male athletes, and many VAERS reports. 

In July 2021 the FDA reported on a potential signal from the BEST system at a time when the VSD had not yet signaled for this outcome. The only way to know if these are adverse reactions caused by the vaccine or not is to focus less on VAERS reports and instead examine the VSD and BEST data. CDC and FDA have the data, systems and knowledge to answer the concerns. Why haven’t they?

Public health officials face a temptation to summarily dismiss anecdotal vaccine injury stories and people concerned about the publicly available VAERS reports, but in public health, we cannot do that. We must take people’s concerns seriously. 

Whatever the truth is, we need to convincingly determine whether there is a problem or not and make that evidence public. Rather than the CDC and FDA feeding the public with inferior VAERS data that cannot answer the question, Americans deserve to be presented with solid evidence from the superior VSD and BEST systems.