We tackle issues that matter to doctors

Ep. 48 Many Statisticians, Many Answers: The Methodological Factor in the Replication Crisis

Brian Nosek, PhD

In 550 BC, the Greek philosopher Heraclitus famously declared: “No man ever steps in the same river twice, for it’s not the same river and he’s not the same man.”  In this episode, we learn from our guest whether scientists can step into the same data pool and obtain the same research results twice.

Brian Nosek is Professor of Psychology at the University of Virginia.  He is also the co-founder and Executive Director of the Center for Open Science, an organization dedicated to fostering transparency and collaboration in scientific research.

In 2015, Professor Nosek and his team published in the journal Science a widely acclaimed and widely discussed paper that shed light on the extent to which psychological research findings may not be reproducible when the research is conducted anew.

More recently, his Center conducted a unique project where a single data set was sent to be analyzed by about 30 independent teams of statisticians for the purpose of answering a single question.  The variability in the methods chosen and in the answers obtained was also perhaps sobering, if not perplexing.

GUEST:

Brian Nosek, PhD.  Profile and Twitter

LINKS:

Silberzahn R, Uhlmann EL, Martin DP et al. Many analysts, one dataset: Making transparent how variations in analytical choices affect results. (2018, Advances in Methods and Science in Psychological Research, open access pre-print here)

Klein RA, Vianello M, Hasselman F, et alMany Labs 2: Investigating Variation in Replicability Across Sample and Setting. (2018, open access pre-print here)

Open Science Collaboration. Estimating the reproducibility of psychological science. (2015 in Science, open access here)

WATCH ON YOUTUBE:

There is no video available for this episode.

Leave a Comment