Debunking decreases mistaken causal inferences: A direct replication of Irving et al. (2022)
DOI:
https://doi.org/10.17879/replicationresearch-2026-10085Keywords:
Causal inferences, Debunking, Statistical misinformation, Reproducibility, ReplicationAbstract
We conducted a direct replication of Irving et al. (2022) examining whether a debunking statement reduces mistaken causal inferences from correlational evidence. We randomly allocated participants to read a news article containing either a correction statement (n = 65) or no-correction (n = 64). Participants in the correction group made fewer causal inferences (δ = 0.65, 95% HDI [0.29, 1.00]) and more correlational inferences (δ = 0.72, 95% HDI [0.36, 1.08]) compared to the no-correction group. Our exploratory analyses also supported the robustness of their findings to alternative modelling choices. Our findings represent a successful direct replication of Irving et al.
References
Adams, R. C., Challenger, A., Bratton, L., Boivin, J., Bott, L., Powell, G., Williams, A., Chambers, C. D., & Sumner, P. (2019). Claims of causality in health news: A randomised trial. BMC Medicine, 17(1), Article 91. https://doi.org/10.1186/s12916-019-1324-7
Alister, M., Vickers-Jones, R., Sewell, D. K., & Ballard, T. (2021). How do we choose our giants? Perceptions of replicability in psychological science. Advances in Methods and Practices in Psychological Science, 4(2), Article 25152459211018199. https://doi.org/10.1177/25152459211018199
Aust, F., & Barth, M. (2023). papaja: Prepare reproducible APA journal articles with R Markdown [Computer software]. https://github.com/crsh/papaja
Ben-Shachar, M. S., Lüdecke, D., & Makowski, D. (2020). effectsize: Estimation of effect size indices and standardized parameters. Journal of Open Source Software, 5(56), Article 2815. https://doi.org/10.21105/joss.02815
Bürkner, P.-C. (2017). brms: An R package for Bayesian multilevel models using Stan. Journal of Statistical Software, 80(1), 1–28. https://doi.org/10.18637/jss.v080.i01
Champely, S. (2020). pwr: Basic functions for power analysis [Computer software]. https://CRAN.R-project.org/package=pwr
Chan, M.-P. S., Jones, C. R., Hall Jamieson, K., & Albarracín, D. (2017). Debunking: A meta-analysis of the psychological efficacy of messages countering misinformation. Psychological Science, 28(11), 1531–1546. https://doi.org/10.1177/0956797617714579
Clarke, B., Alley, L. J., Ghai, S., Flake, J. K., Rohrer, J. M., Simmons, J. P., Schiavone, S. R., & Vazire, S. (2024). Looking our limitations in the eye: A call for more thorough and honest reporting of study limitations. Social and Personality Psychology Compass, 18(7), Article e12979. https://doi.org/10.1111/spc3.12979
Clarke, B., Lee, P. Y., Schiavone, S. R., Rhemtulla, M., & Vazire, S. (2026). The prevalence of direct replication articles in top-ranking psychology journals. American Psychologist, 81(2), 238–253. https://doi.org/10.1037/amp0001385
Clifford, S., & Jerit, J. (2014). Is there a cost to convenience? An experimental comparison of data quality in laboratory and online studies. Journal of Experimental Political Science, 1(2), 120–131. https://doi.org/10.1017/xps.2014.5
Delaney, C. (2005). The Spirituality Scale: Development and psychometric testing of a holistic instrument to assess the human spiritual dimension. Journal of Holistic Nursing, 23(2), 145–167. https://doi.org/10.1177/0898010105276180
Ecker, U. K. H., & Antonio, L. M. (2021). Can you believe it? An investigation into the impact of retraction source credibility on the continued influence effect. Memory & Cognition, 49(4), 631–644. https://doi.org/10.3758/s13421-020-01129-y
Ecker, U. K. H., Lewandowsky, S., Cook, J., Schmid, P., Fazio, L. K., Brashier, N., Kendeou, P., Vraga, E. K., & Amazeen, M. A. (2022). The psychological drivers of misinformation belief and its resistance to correction. Nature Reviews Psychology, 1(1), 13–29. https://doi.org/10.1038/s44159-021-00006-y
Fasce, A., Rosales-Trabuco, J., Barberia, I., Pinedo-Yzaguirre, E., Espinoza Nuñez, J. M., Ortiz-Treviños, W. M., Pizarro-Moncada, C. D., Reyes-Bossio, M., Carbajal-León, C., & Rodríguez-Ferreiro, J. (2026). Effects of debunking interventions on endorsement of alternative medicine: A randomized controlled experiment in Peru. Scientific Reports, 16(1), 4995. https://doi.org/10.1038/s41598-026-38260-w
Field, S. M., Hoekstra, R., Bringmann, L., & van Ravenzwaaij, D. (2019). When and why to replicate: As easy as 1, 2, 3? Collabra: Psychology, 5(1), Article 46. https://doi.org/10.1525/collabra.218
Firke, S. (2023). janitor: Simple tools for examining and cleaning dirty data [Computer software]. https://CRAN.R-project.org/package=janitor
Gabry, J., Simpson, D., Vehtari, A., Betancourt, M., & Gelman, A. (2019). Visualization in Bayesian workflow. Journal of the Royal Statistical Society A (Statistics in Society), 182(2), 389–402. https://doi.org/10.1111/rssa.12378
Ghai, S., Forscher, P. S., & Chuan-Peng, H. (2024). Big-team science does not guarantee generalizability. Nature Human Behaviour, 8(6), 1053–1056. https://doi.org/10.1038/s41562-024-01902-y
Ginn, J., O’Brien, J., & Silge, J. (2022). qualtRics: Download ‘Qualtrics’ survey data [Computer software]. https://CRAN.R-project.org/package=qualtRics
Hameleers, M. (2023). Disinformation as a context-bound phenomenon: Toward a conceptual clarification integrating actors, intentions and techniques of creation and dissemination. Communication Theory, 33(1), 1–10. https://doi.org/10.1093/ct/qtac021
Irving, D., Clark, R. W. A., Lewandowsky, S., & Allen, P. J. (2022). Correcting statistical misinformation about scientific findings in the media: Causation versus correlation. Journal of Experimental Psychology: Applied, 28(1), 1–9. https://doi.org/10.1037/xap0000408
Isager, P. M., van Aert, R. C. M., Bahník, Š., Brandt, M. J., DeSoto, K. A., Giner-Sorolla, R., Krueger, J. I., Perugini, M., Ropovik, I., van ’t Veer, A. E., Vranka, M., & Lakens, D. (2023). Deciding what to replicate: A decision model for replication study selection under resource and knowledge constraints. Psychological Methods, 28(2), 438–451. https://doi.org/10.1037/met0000438
Jang, J., Lee, E.-J., & Shin, S. Y. (2019). What debunking of misinformation does and doesn’t. Cyberpsychology, Behavior, and Social Networking, 22(6), 423–427. https://doi.org/10.1089/cyber.2018.0608
Kay, M. (2024). tidybayes: Tidy data and geoms for Bayesian models [Computer software]. https://doi.org/10.5281/zenodo.1308151
Klein, R. A., Vianello, M., Hasselman, F., Adams, B. G., Adams, R. B., Alper, S., Aveyard, M., Axt, J. R., Babalola, M. T., Bahník, Š., Batra, R., Berkics, M., Bernstein, M. J., Berry, D. R., Bialobrzeska, O., Binan, E. D., Bocian, K., Brandt, M. J., Busching, R., … Nosek, B. A. (2018). Many Labs 2: Investigating variation in replicability across samples and settings. Advances in Methods and Practices in Psychological Science, 1(4), 443–490. https://doi.org/10.1177/2515245918810225
LeBel, E. P., McCarthy, R. J., Earp, B. D., Elson, M., & Vanpaemel, W. (2018). A unified framework to quantify the credibility of scientific findings. Advances in Methods and Practices in Psychological Science, 1(3), 389–402. https://doi.org/10.1177/2515245918787489
LeBel, E. P., Vanpaemel, W., Cheung, I., & Campbell, L. (2019). A brief guide to evaluate replications. Meta-Psychology, 3. https://doi.org/10.15626/MP.2018.843
Lee, Y.-I., Mu, D., Hsu, Y.-C., Wojdynski, B. W., & Binford, M. (2024). Misinformation or hard to tell? An eye-tracking study to investigate the effects of food crisis misinformation on social media engagement. Public Relations Review, 50(4), Article 102483. https://doi.org/10.1016/j.pubrev.2024.102483
Lenth, R. V. (2023). emmeans: Estimated marginal means, aka least-squares means [Computer software]. https://CRAN.R-project.org/package=emmeans
Lewandowsky, S., Ecker, U. K. H., Cook, J., van der Linden, S., Roozenbeek, J., Oreskes, N., & McIntyre, L. C. (2024). Liars know they are lying: Differentiating disinformation from disagreement. Humanities and Social Sciences Communications, 11(1), Article 986. https://doi.org/10.1057/s41599-024-03503-6
Lewandowsky, S., Ecker, U. K. H., Seifert, C. M., Schwarz, N., & Cook, J. (2012). Misinformation and its correction: Continued influence and successful debiasing. Psychological Science in the Public Interest, 13(3), 106–131. https://doi.org/10.1177/1529100612451018
Loomba, S., de Figueiredo, A., Piatek, S. J., de Graaf, K., & Larson, H. J. (2021). Measuring the impact of COVID-19 vaccine misinformation on vaccination intent in the UK and USA. Nature Human Behaviour, 5(3), 337–348. https://doi.org/10.1038/s41562-021-01056-1
Lorko, M., Čavojová, V., Šrol, J., Priesol, R., Jalakšová, P., & Tužilová, B. (2026). Timing matters: The effects of prebunking versus debunking on trust in disinformation. Personality and Social Psychology Bulletin. Advance online publication.. https://doi.org/10.1177/01461672251411571
Lüdecke, D., Ben-Shachar, M. S., Patil, I., Waggoner, P., & Makowski, D. (2021). performance: An R package for assessment, comparison and testing of statistical models. Journal of Open Source Software, 6(60), Article 3139. https://doi.org/10.21105/joss.03139
Makel, M. C., Plucker, J. A., & Hegarty, B. (2012). Replications in psychology research: How often do they really occur? Perspectives on Psychological Science, 7(6), 537–542. https://doi.org/10.1177/1745691612460688
Makowski, D., Ben-Shachar, M. S., & Lüdecke, D. (2019). bayestestR: Describing effects and their uncertainty, existence and significance within the Bayesian framework. Journal of Open Source Software, 4(40), Article 1541. https://doi.org/10.21105/joss.01541
Meade, A. W., & Craig, S. B. (2012). Identifying careless responses in survey data. Psychological Methods, 17(3), 437–455. https://doi.org/10.1037/a0028085
Morey, R. D., & Rouder, J. N. (2023). BayesFactor: Computation of Bayes factors for common designs [Computer software]. https://CRAN.R-project.org/package=BayesFactor
Nadelson, L., Jorcyk, C., Yang, D., Jarratt Smith, M., Matson, S., Cornell, K., & Husting, V. (2014). I just don’t trust them: The development and validation of an assessment instrument to measure trust in science and scientists. School Science and Mathematics, 114(2), 76–86. https://doi.org/10.1111/ssm.12051
National Academies of Sciences, Engineering, and Medicine. (2019). Reproducibility and replicability in science. The National Academies Press. https://doi.org/10.17226/25303
Nosek, B. A., & Errington, T. M. (2017). Making sense of replications. eLife, 6, e23383. https://doi.org/10.7554/eLife.23383
Park, S., & Nan, X. (2026). Generative AI and misinformation: A scoping review of the role of generative AI in the generation, detection, mitigation, and impact of misinformation. AI & Society, 41(2), 1501–1515. https://doi.org/10.1007/s00146-025-02620-3
Pedersen, T. L. (2023). patchwork: The composer of plots [Computer software]. https://CRAN.R-project.org/package=patchwork
R Core Team. (2026). R: A language and environment for statistical computing [Computer software]. R Foundation for Statistical Computing. https://doi.org/10.32614/R.manuals
Revelle, W. (2023). psych: Procedures for psychological, psychometric, and personality research [Computer software]. https://CRAN.R-project.org/package=psych
Rouder, J. N., Speckman, P. L., Sun, D., Morey, R. D., & Iverson, G. (2009). Bayesian t tests for accepting and rejecting the null hypothesis. Psychonomic Bulletin & Review, 16(2), 225–237. https://doi.org/10.3758/PBR.16.2.225
Schmid, P. (2026). Debunking health misinformation with empathy. Current Opinion in Psychology, 67, Article 102213. https://doi.org/10.1016/j.copsyc.2025.102213
Sumner, P., Vivian-Griffiths, S., Boivin, J., Williams, A., Venetis, C. A., Davies, A., Ogden, J., Whelan, L., Hughes, B., Dalton, B., Boy, F., & Chambers, C. D. (2014). The association between exaggeration in health related science news and academic press releases: Retrospective observational study. BMJ, 349, g7015. https://doi.org/10.1136/bmj.g7015
The Turing Way Community. (2025). The Turing Way: A handbook for reproducible, ethical and collaborative research. Zenodo. https://doi.org/10.5281/zenodo.15213042
van de Schoot, R., Depaoli, S., King, R., Kramer, B., Märtens, K., Tadesse, M. G., Vannucci, M., Gelman, A., Veen, D., Willemsen, J., & Yau, C. (2021). Bayesian statistics and modelling. Nature Reviews Methods Primers, 1(1), Article 1. https://doi.org/10.1038/s43586-020-00001-2
Wickham, H. (2016). ggplot2: Elegant graphics for data analysis. Springer. https://doi.org/10.1007/978-3-319-24277-4
Wickham, H., Averick, M., Bryan, J., Chang, W., McGowan, L. D., François, R., Grolemund, G., Hayes, A., Henry, L., Hester, J., Kuhn, M., Pedersen, T. L., Miller, E., Bache, S. M., Müller, K., Ooms, J., Robinson, D., Seidel, D. P., Spinu, V., … Yutani, H. (2019). Welcome to the tidyverse. Journal of Open Source Software, 4(43), Article 1686. https://doi.org/10.21105/joss.01686
Wickham, H., François, R., Henry, L., Müller, K., & Vaughan, D. (2023). dplyr: A grammar of data manipulation [Computer software]. https://CRAN.R-project.org/package=dplyr
Wickham, H., Hester, J., & Bryan, J. (2023). readr: Read rectangular text data [Computer software]. https://CRAN.R-project.org/package=readr
Wickham, H., Vaughan, D., & Girlich, M. (2023). tidyr: Tidy messy data [Computer software]. https://CRAN.R-project.org/package=tidyr
Wilke, C. O. (2020). cowplot: Streamlined plot theme and plot annotations for ‘ggplot2’ [Computer software]. https://CRAN.R-project.org/package=cowplot
Xie, Y. (2015). Dynamic documents with R and knitr (2nd ed.). Chapman & Hall/CRC. https://yihui.org/knitr/
Xu, G., Qian, M., & Meng, L. (2025). Misinformation dissemination on social media: Key research themes and evolutionary paths between 2013 and 2023. Humanities and Social Sciences Communications, 12(1), Article 1775. https://doi.org/10.1057/s41599-025-06067-1
Zwaan, R. A., Etz, A., Lucas, R. E., & Donnellan, M. B. (2018). Making replication mainstream. Behavioral and Brain Sciences, 41, Article e120. https://doi.org/10.1017/S0140525X17001972
Downloads
Published
How to Cite
License
Copyright (c) 2026 Dhruv Aggarwal, Keyi Zhang, James Edward Bartlett
This work is licensed under a Creative Commons Attribution 4.0 International License.
