The Healthy User Effect

“The Broccoli Study”

I got this comment on a YouTube video – “What did humans evolve to eat?”

While I’m not sure about the sincerity of the question, I want to go ahead and discuss the problem with linking broccoli consumption and health.

The Study

So lets say I was going to go and do this study for him. I go and find all these people eating broccoli.

And I find that they are leaner and healthier than the rest of the population. Great.

Then, since I want to be a good scientist, I compare them to heavy meat-eaters.

I find the meat-eaters aren’t as lean or healthy and they die earlier than the broccoli-eaters.

The Conclusion

So eat broccoli and don’t eat meat – right?

Not so fast…

This is an example of epidemiology where I conducted an uncontrolled observational study. In particular this study is going to suffer from the “Healthy User Effect” – or sampling bias (among numerous other study limitations).

Since we’ve been told eating broccoli is healthy since it’s loaded with antioxidants and fiber, is it possible these people are more likely to exercise, avoid drinking and smoking, and in general take care of themselves?

We’ve also been told for decades to avoid meat, it causes heart disease and cancer.

So people that eat a lot of meat – is it possible they don’t care what the USDA says – they don’t listen to their doctor, and they are more likely to smoke, drink, not exercise or adhere to anything anyone says?

Confounding Variables

And then…what else are these people eating?

If you eat a heavy meat diet, lets say 50% of your calories, that’d be considered a good amount of meat today, but the other 50% of calories are from beer and bread – that doesn’t tell me anything about meat.

With epidemiology we have infinite variables – and we don’t try and control for them, which is ok.

A common larger variable is this “Healthy User Effect” – which you have to keep an eye on.

The Point

The point of epidemiology is:

1. You have a hypothesis
2. You investigate with association from epidemiology
3. If you find significant, strong correlation than move on to intelligently designed randomized controlled clinical trials

And guess what, the vast majority of epidemiological conclusions do not hold up under clinical trials.

I’m not saying epidemiology is useless, especially if its put up against Hill’s criteria, but you have to understand it for what it is and the vast limitations of such studies. The Healthy User Effect is a common variable that has to be taken into account with evaluating epidemiology.

So although I won’t be conducing a Broccoli Study anytime soon, I do appreciate the questions and comments 🙂