An Investigation of the NeurIPS and ICML 2025 Position Tracks

ML venues shape what kinds of research claims become legible to reviewers and what forms of evidence count as rigorous. The NeurIPS and ICML Position Paper Tracks were created for agenda-setting work, making their early composition worth auditing. $\mathbf{\text{This}}\mathbf{\text{paper}}\mathbf{\text{argues}}\mathbf{\text{that}}\mathbf{\text{the}}\mathbf{\text{publicly}}\mathbf{\text{accessible}}\mathbf{\text{2025}}\mathbf{\text{reviewed}}\mathbf{\text{pool}}\mathbf{\text{is}}\mathbf{\text{dominated}}\mathbf{\text{by}}\mathbf{\text{reformist}}\mathbf{\text{critique,}}\mathbf{\text{and}}\mathbf{\text{that}}\mathbf{\text{the}}\mathbf{\text{track}}\mathbf{\text{should}}\mathbf{\text{explicitly}}\mathbf{\text{solicit}}\mathbf{\text{direction-setting}}\mathbf{\text{work}}\mathbf{\text{alongside,}}\mathbf{\text{not}}\mathbf{\text{in}}\mathbf{\text{place}}\mathbf{\text{of,}}\mathbf{\text{the}}\mathbf{\text{reformist}}\mathbf{\text{critiques}}\mathbf{\text{it}}\mathbf{\text{already}}\mathbf{\text{hosts}}\mathbf{\text{well.}}$ We audit every accessible submission to the NeurIPS 2025 and ICML 2025 Position Tracks under a pre-specified rubric, and compare the resulting pattern with a reference class of widely recognized agenda-shifting ML papers. Three-quarters of audited submissions critique an existing benchmark, evaluation, or methodology; these papers score highly on our artifact-coupling rubric, but evidentiary depth does not predict reviewer rating. The reference class (AlexNet, the Transformer, Concrete Problems in AI Safety, and others) differs from the accessible reviewed pool in \emph{artifact kind}: agenda-shifting papers typically gave the field something new to build on, test against, or contest, such as a measurement protocol, benchmark proposal, toy implementation, dataset card, audit template, or falsifiable experimental program. We close with four CFP-level interventions aimed at broadening the submission mix without displacing the critiques the track already hosts well.