Neural NLI models overfit dataset artifacts instead of truly reasoning. A hypothesis-only model gets 57.7% in SNLI, showing strong spurious correlations, and 38.6% of the baseline errors are the result of these artifacts. We propose Product-of-Experts (PoE) training, which downweights examples where biased models are overconfident. PoE nearly preserves accuracy (89.10% vs. 89.30%) while cutting bias reliance by 4.71% (bias agreement 49.85% to 45%). An ablation finds lambda = 1.5 that best balances debiasing and accuracy. Behavioral tests still reveal issues with negation and numerical reasoning.
@article{arxiv.2604.19069,
title = {Product-of-Experts Training Reduces Dataset Artifacts in Natural Language Inference},
author = {Aby Mammen Mathew},
journal= {arXiv preprint arXiv:2604.19069},
year = {2026}
}
Comments
10 pages, 3 figures, 4 tables. Single-author paper