Why Observational Studies Lie to You

If you’ve spent five minutes paying attention to medical news, you’ve undoubtedly seen the headlines: “People who take statins have 30% fewer heart attacks!” or “Breast cancer screening makes women live longer!” or “Menopausal Hormone therapy lowers Alzheimer’s risk.”

These all sound wonderful, don’t they? Just swallow a daily pill, or submit to regular screening, and you’ll join the ranks of the immortal. Were only life so simple. 

Yet before you rush off to demand a prescription, let me introduce you to one of the sneakiest, most persistent mirages from the world of medical research, which we call the “Healthy User Bias.”

Readers of Brownstone were recently introduced to this concept related to the Covid vaccines in an article entitled “A Hall of Mirrors of the Healthy Vaccinee Effect,” which demonstrated that those who were vaccinated were not the same as those who weren’t. Very frail people and those close to death were less likely to get vaccinated, which is understandable, and so those who didn’t get the jab died at a higher rate than those that did. Simple, right?

Understanding this single concept may not, overnight, elevate your medical BS detector from “gullible consumer” to “battle-tested skeptic,” but to be fair, it might help. 

The Kale-Eating, 6 AM Jogger Effect, and the Placebo Effect

What is the Healthy User Bias? It’s embarrassingly simple. In observational studies—where researchers merely watch what people do in the real world rather than running an experiment—the people who voluntarily take preventive drugs or pop daily supplements are fundamentally different from those who don’t. No surprise there, right? 

On average, “healthy users” are wealthier, better educated, and more health-obsessed. They are seriously more obedient when it comes to vaccines (including the Covid vaccine). These sorts of people want to do the best they can within their powers, which includes things that involve some friction: they eat organic kale, jog at dawn, wear seatbelts, don’t smoke, and visit their doctors regularly. They love tests of all kinds. 

A 2006 meta-analysis quantified the healthy user effect (which they called the healthy adherer effect) across 21 studies. What they found is mind-blowing in its implications. It found that those who religiously took their prescribed pills (which we call ‘good adherence’), even if it was a placebo, showed a 44% reduction in mortality (which was identical to adherence to proven beneficial therapy). All this shows is that the obedient, healthy people are more likely to do what most physicians want: they take their medication as ordered by their doctors.

When researchers observe people over ten years and notice they have fewer heart attacks and fewer strokes the light goes on. Hey, they might live longer! This is where pharmaceutical cheerleaders jump up and down, declaring: “Look! It was our pill!”

No. Sorry, it most likely wasn’t the pill. It was the person who took the pill. It was the fact that the patient was already doing 27 zillion other things to stay healthy. Maybe they had a gene that said: don’t do stuff to excess that are known to be bad for you (ie: smoking, drinking, committing violent crimes, extreme sports, being grossly overweight, being a couch potato, etc)

The pill was just along for the ride. Observational research routinely mistakes a healthy lifestyle for pharmaceutical magic. 

By now you’re saying, Okay Mr. Alansmartypants, gimme some examples. 

The Hormone Replacement Disaster: A Cautionary Tale

This is a complex, decades long story but likely the biggest medical reversal in modern history: Hormone Replacement Therapy (HRT) prescribed for women who lived long enough to savour the nasty symptoms of menopause. 

For decades, observational studies swore that HRT was a fountain-of-youth elixir for menopausal women, claiming a 50% reduction in heart attacks. The recommendations to get every woman of a ‘certain age’ on hormones were ubiquitous. Doctors prescribed it to millions of women around the world, convinced they were doing God’s work in protecting women’s hearts. But they were wrong. 

Why were observational studies of HRT so wildly wrong? Healthy user bias. 

Throughout the 80s and 90s studies on over 100,000 nurses allegedly showed that postmenopausal women who utilized HRT had a significantly lower risk of developing coronary heart disease compared to women who did not use hormone therapy. Because the study was large and followed women over a long period, these results were highly influential, convincing many in the medical community that HRT not only managed menopausal symptoms but also provided a protective, preventative benefit against cardiovascular disease. They claimed that HRT was a lifesaver.

Problem was, the women taking HRT in the 1980s and 90s were wealthier, exercised more, and had better healthcare access than women who didn’t. That was the entirety of the effect. 

Thankfully for all of us, the US government funded the Women’s Health Initiative (WHI)—a massive Randomized Controlled Trial (RCT)—and that’s when the truth exploded. An RCT doesn’t just watch people; it flips a coin and randomly assigns half the women to take HRT and the other half to take a dummy pill (placebo). Randomization completely destroys healthy user bias because both groups end up with equal numbers of joggers, smokers, and couch potatoes.

The WHI trial results were staggering: HRT didn’t prevent heart attacks at all. In fact, it increased the risk of heart attacks, strokes, blood clots, and breast cancer. The “miracle drug” was actually harming women, and the observational “healthy user” illusion had blinded medicine for a generation.

Statins in Healthy People: The Mirage Continues

We see the exact same trick played today with cholesterol-lowering statin drugs in healthy people (“primary prevention”). Observational studies endlessly brag about how statin users do better but when independent groups—like the Therapeutics Initiative at the University of British Columbia—analyzed only randomized trials comparing statins to placebos in healthy people, the magic vanished. In fact, those same researchers found that statin users were less likely to be involved in motor vehicle accidents, an absurd conclusion that completely threw the statin hypothesis under the bus. 

The RCT evidence proves that for otherwise healthy people with no history of heart disease, taking a daily statin provides no proven net health benefit—it does not reduce your overall chance of dying or ending up in the hospital. Yet millions of healthy adults swallow them daily because observational fluff and statistical sleight-of-hand convinced their doctors that a biomarker on a lab report equals staying alive longer.

The Bottom Line: Demand the Coin Flip

Observational studies are a superb playground for pharmaceutical public relations. They are fine for generating hypotheses, but useless for proving that a drug actually works.

Just last month a story in the New York Times claimed that Hormone Therapy reduced the risk of Alzheimer’s disease. To their credit they also added that it was observational research and comes with these sort of selection biases, such as healthy user bias. A NY Times headline is certainly good for the manufacturers of Alzheimer’s drugs, even if the research behind it was an observational study riddled with healthy user bias. 

What’s the takeaway here? When a drug company or headline-hungry journalist tells you a certain long-term drug therapy will prevent disease, never let “healthy user” observational noise influence your decision. Always ask the million-dollar question: “Where is the randomized, placebo-controlled trial?”

A fairly done, randomized comparison strips away the healthy user illusion and tells you whether a drug is actually helping you—or just taking credit for your lifestyle while exposing you to unearned harms.

Until you see reliable RCT evidence, keep your sneakers on, eat your veggies, and treat observational drug claims with a strong and healthy dose of skepticism.