Large language models exhibit strong reasoning capabilities, yet often rely on shortcuts such as surface pattern matching and answer memorization rather than genuine logical inference. We propose Shortcut-Aware Reasoning Training (SART), a gradient-aware framework that detects and mitigates shortcut-promoting samples via ShortcutScore and gradient surgery. Our method identifies shortcut signals through gradient misalignment with validation objectives and answer-token concentration, and modifies training dynamics accordingly. Experiments on controlled reasoning benchmarks show that SART achieves +16.5% accuracy and +40.2% robustness over the strongest baseline, significantly improving generalization under distribution shifts. Code is available at: https://github.com/fuyanjie/short-cut-aware-data-centric-reasoning.
@article{arxiv.2603.20899,
title = {Mitigating Shortcut Reasoning in Language Models: A Gradient-Aware Training Approach},
author = {Hongyu Cao and Kunpeng Liu and Dongjie Wang and Yanjie Fu},
journal= {arXiv preprint arXiv:2603.20899},
year = {2026}
}
Comments
12 pages, 2 figures. Preprint. Experiments on synthetic reasoning benchmarks. Code available