What is the "replication crisis" plaguing medical research - and can we fix it?

When most of us read the results of a study that was conducted by a prestigious group of researchers and published in a respected medical or scientific journal, we trust it as fact.

But there’s a dark side to scientific and medical research: In many cases, the results aren’t the same when other scientists re-create the study. This phenomenon is so widespread that it’s been dubbed “the replication crisis.”

About a decade ago, researchers discovered that a disturbingly large number of published studies in a wide range of fields, including psychology, physics, and medicine, had results that were impossible to replicate.

In a 2016 poll of 1,500 scientists conducted by respected journal Nature, 70% of participants reported having been unable to reproduce the findings of a study at least once. This includes 67% of medical researchers.

Even more troubling, 50% of the poll’s respondents admitted to not being able to reproduce the results of their own research!

The research in question isn’t just small experiments or niche projects. Experts have found that popular studies whose findings are covered in mainstream publications and even sometimes “go viral” often have results that are impossible to reproduce, as well.

Scientific and medical research is supposed to be trustworthy, especially because it seems to go through so much gatekeeping. In order to have research published in the prestigious Journal of American Medicine (JAMA), for instance, researchers face a long list of requirements for everything from manuscript formatting to verifying ethics and reporting.

But the replication crisis almost makes it seem like none of this matters.

So, how did we get here?

There are a number of theories about the cause(s) of the replication crisis. The explanations that are most commonly given are:

- the “publish or perish” mentality. This is the idea that academics must continue to share and publish new findings in order to be relevant. It can push researchers to move quickly, rather than take time to carefully consider and test results.

- the desire for significant, concrete results. Journalist Reginald Davey reports that scientific and academic journals want notable, concrete results, rather than theories or ideas inspired by research. This can lead to coaxing or even forcing a conclusion from results that should have been better tested.

So, is the failure to replicate studies just a given we have to deal with? Should we just take all scientific and medical research with a grain of salt?

You probably feel ill at ease considering this option, and fortunately, so do a lot of scientists and doctors. Many of them are taking measures to ensure that studies can indeed be replicated.

For instance, a number of journals now require “preregistration”, where researchers explain how they’ll conduct their study before conducting it. Peers then review the process for approval.

A recent Vox feature spotlighted a new strategy in the fight against the replication crisis. The Transparent Replications Project is an independent group of scientists who will attempt to replicate any research being submitted for publication.

The project is ambitious and it does have certain limits; for instance, for now it will focus only on psychology research, as this is the field the members are most qualified to work with. That’s a good place to start, since psychology seems to be the area hardest hit by the replication crisis. Still, it leaves a lot of other research unverified for the time being.

Hopefully, these strategies and any other new ideas that might arise, will help scientific and medical research become as trustworthy as most of us believe it to be. Until then, unfortunately, it’s a good idea to take research results with a healthy dose of skepticism.

Image source

Contact Our Writer – Alysa Salzberg