The potential of preregistration in psychology: assessing preregistration producibility and preregistration-study consistency
van den Akker, Olmo, Bakker, Marjan, van Assen, Marcel, Pennington, Charlotte, Verweij, Leone, Elsherif, Mahmoud, Claesen, Aline, Gaillard, Stefan, Yeung, Siu Kit, Frankenberger, Jan-Luca, Krautter, Kai, Evans, Thomas ORCID: 0000-0002-6670-0718 , Heppel, Frederique, Schoch, Sarah, Korbmacher, Max, Yamada, Yuki, Albayrak-Aydemir, Nihan, Alzahawi, Shilaan, Sarafoglou, Alexandra, Sitnikov, Maksim, Dechterenko, Filip, Wingen, Sophia, Grinschgl, Sandra, Hartmann, Helena, Stewart, Suzanne, de Oliveira, Catia, Ashcroft-Jones, Sarah, Baker, Bradley and Wicherts, Jelte (2024) The potential of preregistration in psychology: assessing preregistration producibility and preregistration-study consistency. Psychological Methods. ISSN 1082-989X (Print), 1939-1463 (Online) (In Press)
|
PDF (AAM)
47245_EVANS_The_potential_of_preregistration_in_psychology_Assessing_preregistration_producibility_and_preregistration-study_consistency.pdf - Accepted Version Download (564kB) | Preview |
Abstract
Study preregistration has become increasingly popular in psychology, but its effectiveness in restricting potentially biasing researcher degrees of freedom remains unclear. We used an extensive protocol to assess the producibility (i.e., the degree to which a study can be properly conducted based on the available information) of preregistrations and the consistency between preregistration and their corresponding papers for 300 psychology studies. We found that preregistrations often lack methodological details and that undisclosed deviations from preregistered plans are frequent. Combining the producibility and consistency results highlights that biases due to researcher degrees of freedom are likely in many preregistered studies. More comprehensive registration templates typically yielded more producible and hence better preregistrations. We did not find that effectiveness of preregistrations differed over time or between original and replication studies. Furthermore, we found that operationalizations of variables were generally more effectively preregistered than other study parts. Inconsistencies between preregistrations and published studies were mainly encountered for data collection procedures, statistical models, and exclusion criteria. Our results indicate that, to unlock the full potential of preregistration, researchers in psychology should aim to write more producible preregistrations, adhere to these preregistrations more faithfully, and more transparently report any deviations from their preregistrations. This could be facilitated by training and education to improve preregistration skills, as well as the development of more comprehensive templates.
Item Type: | Article |
---|---|
Uncontrolled Keywords: | preregistration; preregistration deviation; preregistration effectiveness; preregistration template; selective reporting; meta-research |
Subjects: | B Philosophy. Psychology. Religion > BF Psychology |
Faculty / School / Research Centre / Research Group: | Faculty of Education, Health & Human Sciences Faculty of Education, Health & Human Sciences > School of Human Sciences (HUM) |
Last Modified: | 23 May 2024 08:46 |
URI: | http://gala.gre.ac.uk/id/eprint/47245 |
Actions (login required)
View Item |
Downloads
Downloads per month over past year