English

Diffusion Secant Alignment for Score-Based Density Ratio Estimation

Machine Learning 2025-12-11 v3 Machine Learning

Abstract

Estimating density ratios has become increasingly important with the recent rise of score-based and diffusion-inspired methods. However, current tangent-based approaches rely on a high-variance learning objective, which leads to unstable training and costly numerical integration during inference. We propose \textit{Interval-annealed Secant Alignment Density Ratio Estimation (ISA-DRE)}, a score-based framework along diffusion interpolants that replaces the instantaneous tangent with its interval integral, the secant, as the learning target. We show theoretically that the secant is a provably lower variance and smoother target for neural approximation, and also a strictly more general representation that contains the tangent as the infinitesimal limit. To make secant learning feasible, we introduce the \textit{Secant Alignment Identity (SAI)} to enforce self consistency between secant and tangent representations, and \textit{Contraction Interval Annealing (CIA)} to ensure stable convergence. Empirically, this stability-first formulation produces high efficiency and accuracy. ISA-DRE achieves comparable or superior results with fewer function evaluations, demonstrating robustness under large distribution discrepancies and effectively mitigating the density-chasm problem.

Keywords

Cite

@article{arxiv.2509.04852,
  title  = {Diffusion Secant Alignment for Score-Based Density Ratio Estimation},
  author = {Wei Chen and Shigui Li and Jiacheng Li and Jian Xu and Zhiqi Lin and Junmei Yang and Delu Zeng and John Paisley and Qibin Zhao},
  journal= {arXiv preprint arXiv:2509.04852},
  year   = {2025}
}
R2 v1 2026-07-01T05:22:37.889Z