How Scientific is Psychology?

In any branch of science, being able to replicate any kind of research study, whether it be experimental or observational research, is one of the cornerstones of the scientific method.  Though the "soft" scientific research carried out by psychologists or sociologists tend to be more difficult to replicate than in the"harder" sciences such as chemistry or physics,  serious questions still tend to be raised if a study can't be replicated successfully.

Which is what makes The Reproducibility Project: Psychology (RPP) so important.  A crowdsourced metascience project funded by the Center for Open Sciences in Charlottesville, Virginia, the project carried out independent replication of one hundred psychology studies.  Those studies chosen to be replicated were taken from the 2008 issues of three prominent psychology journals—Journal of Personality and Social Psychology, Psychological Science, and Journal of Experimental Psychology: Learning, Memory, and Cognition.   With nearly three hundred participating researchers, the replications were completed and posted online on April 24, 2015.   The project organizers intended the RPP to be one of the most comprehensive tests of psychological research replicability ever undertaken.

Which makes their findings especially worrisome.  Of the one hundred studies tested, only thirty-nine were successfully replicated (showing the same results as the original researchers with an acceptable level of significance).   An additional twenty-four produced findings that were at least "moderately similar" to what the original researchers had reported though they failed to achieve statistical significance.

Reactions to these findings have been mixed so far.   According to Hal Pashler, a cognitive psychologist and the author of one of the studies that was successfully replicated, these findings demonstrate the credibility problem in psychology.  “A lot of working scientists assume that if it’s published, it’s right,” he says. “This makes it hard to dismiss that there are still a lot of false positives in the literature.”    On the other hand,  Stanford University professor Daniele Fanelli, who  studies science bias and misconduct, suggests that these results aren't out of line with what has been found in other sciences.

“From my expectations, these are not bad at all,” Fanelli says. “Though I have spoken to psychologists who are quite disappointed.”    Replication results for cancer and drug studies have been found to be even lower.   Still, the RPP is extremely ambitious and questions have been raised about how feasible it is to replicate some psychology studies, especially when they involve factors that aren't easily replicated under experimental conditions.   While the researchers attempted to follow the original study as closely as possible, substitutions still needed to be made in many cases which may have affected results.

Greg Hajcak of Stony Brook University raised concerns about some of the rigid criteria adopted by the RPP.   His own study, which looked at how making mistakes affected physiological responses, wasn't fully replicated though researchers found non-significant results that resembled what Hajcak had reported.  “What does it mean to say it’s not a replication, if you have the exact same pattern of results?”, Hajcak asks.

One things that supporters and critics of the RPP results agree on is that more work needs to be done to promote replication studies in psychology journals.   How this will address the ongoing credibility problem of psychology is something future psychologists will need to take into account.

 

 

 

 

 

 

 

 

           

Related Stories

  • What Do Women Look For in Choosing Mates?
  • Unfalsifiability and Belief
  • When Men Face Sexual Harassment
 

 
disclaimer

The information provided on the PsyWeb.com is designed to support, not replace, the relationship that exists between a patient/site visitor and his/her health professional. This information is solely for informational and educational purposes. The publication of this information does not constitute the practice of medicine, and this information does not replace the advice of your physician or other health care provider. Neither the owners or employees of PsyWeb.com nor the author(s) of site content take responsibility for any possible consequences from any treatment, procedure, exercise, dietary modification, action or application of medication which results from reading this site. Always speak with your primary health care provider before engaging in any form of self treatment. Please see our Legal Statement for further information.

PsyWeb Poll

Are you currently taking or have you ever been prescribed anti-depressants?
Yes
50%
No
50%
Total votes: 3979