English

AdaJudge: Adaptive Multi-Perspective Judging for Reward Modeling

Computation and Language 2026-01-14 v1 Machine Learning

Abstract

Reward modeling is essential for aligning large language models with human preferences, yet predominant architectures rely on a static pooling strategy to condense sequences into scalar scores. This paradigm, however, suffers from two key limitations: a static inductive bias that misaligns with task-dependent preference signals, and a representational mismatch, as the backbone is optimized for generation rather than fine-grained discrimination. To address this, we propose AdaJudge, a unified framework that jointly adapts representation and aggregation. AdaJudge first refines backbone representations into a discrimination-oriented space via gated refinement blocks. It then replaces the static readout with an adaptive multi-view pooling module that dynamically routes and combines evidence. Extensive experiments on RM-Bench and JudgeBench show that AdaJudge outperforms strong off-the-shelf reward models and traditional pooling baselines.

Keywords

Cite

@article{arxiv.2601.08097,
  title  = {AdaJudge: Adaptive Multi-Perspective Judging for Reward Modeling},
  author = {Yongliang Miao and Yangyang Liang and Mengnan Du},
  journal= {arXiv preprint arXiv:2601.08097},
  year   = {2026}
}
R2 v1 2026-07-01T09:01:53.431Z