I’ve heard way too many times that observational studies are flawed, and to really confirm a hypothesis you have to do randomized controlled trials. Indeed, this was an argument in the hormone replacement therapy (HRT) controversy (scroll down for the article). Now that I’ve worked with both observational and randomized data, here are a few observations:
- The choice of observational vs. randomized is an important, but not the only, study design choice.
Studies have lots of different design choices: followup length, measurement schedule, when during disease course to observe, assumptions about risk groups, assumptions about stability of risk over time (which was important in the HRT discussion about breast cancer), and the list goes on. A well-designed observational trial can give a lot of more valid information than a poorly-designed randomized trial. - Only one aspect of a randomized trial is randomized (usually). Covariates and subgroups are not randomized.
- Methods exists to make valid comparisons in an observational study. While data have to be handled much more carefully, and assumptions behind the statistical methods have to be examined more carefully. However, very powerful methods such as causal analysis or case-control studies can be used to make strong conclusions.
Observational studies can complement or replace randomized designs. In fact, in controversies such as the use of thimerosol in vaccines, observational studies have been required to supply all the evidence (randomizing children to thimerosol and non-thimerosol groups in a randomized study to see if they develop autism is not ethical). In post-marketing research and development for drugs, observational studies are used to further establish safety, determine the rate of rare serious adverse events, and determine the effects of real-world usage on the efficacy that has been established through randomized trials.
Through careful planning, observational studies can generate new results, extend the results of randomized trials, or even set up new randomized trials.
