Optimal Decision Rules for Composite Binary Hypothesis Testing under Neyman-Pearson Framework
Abstract
The composite binary hypothesis testing problem within the Neyman-Pearson framework is considered. The goal is to maximize the expectation of a nonlinear function of the detection probability, integrated with respect to a given probability measure, subject to a false-alarm constraint. It is shown that each power function can be realized by a generalized Bayes rule that maximizes an integrated rejection probability with respect to a finite signed measure. For a simple null hypothesis and a composite alternative, optimal single-threshold decision rules based on an appropriately weighted likelihood ratio are derived. The analysis is extended to composite null hypotheses, including both average and worst-case false-alarm constraints, resulting in modified optimal threshold rules. Special cases involving exponential family distributions and numerical examples are provided to illustrate the theoretical results.
Cite
@article{arxiv.2505.17851,
title = {Optimal Decision Rules for Composite Binary Hypothesis Testing under Neyman-Pearson Framework},
author = {Yanglei Song and Berkan Dulek and Sinan Gezici},
journal= {arXiv preprint arXiv:2505.17851},
year = {2025}
}