Kekecs, Zoltan, Palfi, Bence, Szaszi, Baranabas, Szecsi, Peter, Zrubka, Mark, Kovacs, Marton, Bakos, Bence, Cousineau, Denis, Tressoldi, Patrizio, Schmidt, Kathleen, Grassi, Massimo, Evans, Thomas ORCID: 0000-0002-6670-0718 , Yamada, Yuki, Miller, Jeremy, Liu, Huanxu, Yonemitsu, Fumiya, Dubrov, Dmitrii, Röer, Jan, Becker, Marvin, Schnepper, Roxane, Ariga, Atsunori, Arriaga, Patricia, Oliveira, Raquel, Põldver, Nele, Kreegipuu, Kairi, Hall, Braeden, Wiechert, Sera, Verschuere, Bruno, Girán, Kyra and Aczel, Balazs (2023) Raising the value of research studies in psychological science by increasing the credibility of research reports: the transparent Psi project. Royal Society Open Science, 10 (2):191375. ISSN 2054-5703 (Online) (doi:https://doi.org/10.1098/rsos.191375)

Preview	PDF (Open Access Article) 38220_EVANS_Raising_the_value_of_research_studies_in_psychological_science_(OA)_2023.pdf - Published Version Available under License Creative Commons Attribution. Download (971kB) | Preview
Preview	PDF (AAM) 38220_EVANS_Raising_the_value_of_research_studies_in_psychological_science.pdf - Accepted Version Available under License Creative Commons Attribution. Download (867kB) | Preview

Abstract

The low reproducibility rate in social sciences has produced hesitation among researchers in accepting published findings at their face value. Despite the advent of initiatives to increase transparency in research reporting, the field is still lacking tools to verify the credibility of research reports. In the present paper, we describe methodologies that let researchers craft highly credible research and allow their peers to verify this credibility. We demonstrate the application of these methods in a multi-lab replication of Bem’s Experiment 1 (2011) on extrasensory perception (ESP), which was co-designed by a consensus panel including both proponents and opponents of Bem’s original hypothesis. In the study we applied direct data deposition in combination with born-open data and real-time research reports to extend transparency to protocol delivery and data collection. We also used piloting, checklists, laboratory logs and video documented trial sessions to ascertain as-intended protocol delivery, and external research auditors to monitor research integrity. We found 49.89% successful guesses, while Bem reported 53.07% success rate, with the chance level being 50%. Thus, Bem’s findings were not replicated in our study. In the paper we discuss the implementation, feasibility, and perceived usefulness of the credibility-enhancing methodologies used throughout the project.

Item Type:	Article
Uncontrolled Keywords:	metascience; research methods; credibility; transparency; replication; real-time procedures
Subjects:	B Philosophy. Psychology. Religion > BF Psychology
Faculty / School / Research Centre / Research Group:	Faculty of Education, Health & Human Sciences Faculty of Education, Health & Human Sciences > School of Human Sciences (HUM)
Last Modified:	07 Feb 2023 16:47
URI:	http://gala.gre.ac.uk/id/eprint/38220

Actions (login required)

View Item

Downloads