How to Evaluate Health Risks

by Maia Szalavitz at the Statistical Assessment Service:

• When you see a news report about new research, it’s important to determine whether the research was experimental or observational.
• If you don't see the words "randomization," "controlled" or "experimental," it's probably an observational study.
• Observational research can never show that one thing causes another; it can only show that there is a "correlation," or association, between two or more things.
• If experimental studies exist, they are almost always better evidence than observational data and should be given greater weight in your decisions about changing your behavior.
• Experimental studies can produce misleading results if they only draw on a few subjects. The more subjects there are in the study, the less likely the results are due to chance or error.
• Where there is data, the best kind of study to base behavior change on is a systematic review of randomized, controlled trials.
• The second best is a randomized controlled trial.
• The third is an observational study.
• When you look at health research and risk determinations, check the dosage given to the subjects and the other related variables to be sure it’s comparable to the kind of exposure you may have or be considering and weigh human data more heavily than animal research.
• If a new study was done on a group that is very different from you and there is no research on more diverse populations, you should give it less weight than one which looks at people who are more similar to you or one which has a more diverse sample.
• Risk increases of less than 200% (a factor of two) are often due to chance. If an observational study links something to a 30% increased risk of cancer, it may sound alarming, but unless there is additional evidence, most scientists do not consider the risk significant enough to prompt a major change in behavior.
• Anecdotes, no matter how compelling, are not ever enough to establish a causal relationship.
• When deciding whether research warrants a change, keep in mind whether it was published in a peer-reviewed journal, as well as who funded it, and what their larger agenda may be.
• New health data picked up by the media is something you should pay attention to, but is almost never something you should build your life around.