Position: Machine Learning Conferences Should Establish a "Refutations and Critiques" Track
Abstract
Science progresses by iteratively advancing and correcting humanity's understanding of the world. In machine learning (ML) research, rapid advancements have led to an explosion of publications, but have also led to misleading, incorrect, flawed or perhaps even fraudulent studies being accepted and sometimes highlighted at ML conferences due to the fallibility of peer review. While such mistakes are understandable, ML conferences do not offer robust processes to help the field systematically correct when such errors are made. This position paper argues that ML conferences should establish a dedicated "Refutations and Critiques" (R&C) Track. This R&C Track would provide a high-profile, reputable platform to support vital research that critically challenges prior research, thereby fostering a dynamic self-correcting research ecosystem. We discuss key considerations including track design, review principles, potential pitfalls, and provide an illustrative example submission concerning a recent ICLR 2025 Oral. We conclude that ML conferences should create official, reputable mechanisms to help ML research self-correct.
Keywords
Cite
@article{arxiv.2506.19882,
title = {Position: Machine Learning Conferences Should Establish a "Refutations and Critiques" Track},
author = {Rylan Schaeffer and Joshua Kazdan and Yegor Denisov-Blanch and Brando Miranda and Matthias Gerstgrasser and Susan Zhang and Andreas Haupt and Isha Gupta and Elyas Obbad and Jesse Dodge and Jessica Zosa Forde and Francesco Orabona and Sanmi Koyejo and David Donoho},
journal= {arXiv preprint arXiv:2506.19882},
year = {2025}
}