English
Related papers

Related papers: Optimal Hypothesis Testing Based on Information Th…

200 papers

The problem of simultaneously testing the marginal distributions of sequentially monitored, independent data streams is considered. The decisions for the various testing problems can be made at different times, using data from all streams,…

Methodology · Statistics 2023-04-21 Yiming Xing , Georgios Fellouris

We propose a frequentist testing procedure that maintains a defined coverage and is optimal in the sense that it gives maximal power to detect deviations from a null hypothesis when the alternative to the null hypothesis is sampled from a…

Applications · Statistics 2020-07-07 Christian Bartels , Johanna Mielke , Ekkehard Glimm

We consider Bayesian multiple hypothesis problem with independent and identically distributed observations. The classical, Sanov's theorem-based, analysis of the error probability allows one to characterize the best achievable error…

Information Theory · Computer Science 2021-11-01 Hüseyin Afşer

This paper is concerned with a Bayesian approach to testing hypotheses in statistical inverse problems. Based on the posterior distribution $\Pi \left(\cdot |Y = y\right)$, we want to infer whether a feature $\langle\varphi,…

Statistics Theory · Mathematics 2025-03-25 Remo Kretschmann , Frank Werner

Prediction sets can wrap around any ML model to cover unknown test outcomes with a guaranteed probability. Yet, it remains unclear how to use them optimally for downstream decision-making. Here, we propose a decision-theoretic framework…

Machine Learning · Statistics 2026-02-10 Tao Wang , Edgar Dobriban

Phishing attacks have become a serious and challenging issue for detection, explanation, and defense. Despite more than a decade of research on phishing, encompassing both technical and non-technical remedies, phishing continues to be a…

Cryptography and Security · Computer Science 2024-04-18 Van Nguyen , Tingmin Wu , Xingliang Yuan , Marthie Grobler , Surya Nepal , Carsten Rudolph

In this paper, we consider sequential testing over a single-sensor, a single-decision center setup. At each time instant $t$, the sensor gets $k$ samples $(k>0)$ and describes the observed sequence until time $t$ to the decision center over…

Information Theory · Computer Science 2021-09-20 Sadaf Salehkalaibar , Vincent Y. F. Tan

We consider the problem of hypotheses testing with the basic simple hypothesis: observed sequence of points corresponds to stationary Poisson process with known intensity. The alternatives are stationary self-exciting point processes. We…

Statistics Theory · Mathematics 2009-03-27 Serguei Dachian , Yury A. Kutoyants

Binary classification is a task that involves the classification of data into one of two distinct classes. It is widely utilized in various fields. However, conventional classifiers tend to make overconfident predictions for data that…

Machine Learning · Computer Science 2025-03-13 Shoma Yokura , Akihisa Ichiki

Reliable estimation of predictive uncertainty is crucial for machine learning applications, particularly in high-stakes scenarios where hedging against risks is essential. Despite its significance, there is no universal agreement on how to…

Machine Learning · Computer Science 2025-06-17 Kajetan Schweighofer , Lukas Aichberger , Mykyta Ielanskyi , Sepp Hochreiter

This article extends the hypotheses assessment method to the case with two competing simple hypotheses. In doing so we further clarify the benefits that hypotheses assessments can bring to classical statistical analyses. Given that…

Methodology · Statistics 2025-03-25 Graham N. Bornholt

It is difficult to discriminate between the state of initial time $t_0$ and that of time $t_0+\Delta t$. The task of discriminating between the two states can be interpreted as the hypothesis testing problem as to a state $\rho$ deciding…

Quantum Physics · Physics 2017-12-04 Keiji Matsumoto , Susumu Osawa

This paper is a review of a particular approach to the method of maximum entropy as a general framework for inference. The discussion emphasizes the pragmatic elements in the derivation. An epistemic notion of information is defined in…

Data Analysis, Statistics and Probability · Physics 2021-08-04 Ariel Caticha

The existence of a (p-)optimal propositional proof system is a major open question in (proof) complexity; many people conjecture that such systems do not exist. Krajicek and Pudlak (1989) show that this question is equivalent to the…

Computational Complexity · Computer Science 2010-02-03 Edward A. Hirsch , Dmitry Itsykson

Simulation-based inference enables learning the parameters of a model even when its likelihood cannot be computed in practice. One class of methods uses data simulated with different parameters to infer models of the likelihood-to-evidence…

Machine Learning · Computer Science 2022-06-08 Giulio Isacchini , Natanael Spisak , Armita Nourmohammad , Thierry Mora , Aleksandra M. Walczak

Information theory has been very successful in obtaining performance limits for various problems such as communication, compression and hypothesis testing. Likewise, stochastic control theory provides a characterization of optimal policies…

Information Theory · Computer Science 2018-10-15 Dhruva Kartik , Ekraam Sabir , Urbashi Mitra , Prem Natarajan

The severity of type II errors is frequently ignored when deriving a multiple testing procedure, even though utilizing it properly can greatly help in making correct decisions. This paper puts forward a theory behind developing a multiple…

Methodology · Statistics 2014-03-25 Li He , Sanat K. Sarkar , Zhigen Zhao

In this paper, we introduce a flexible and widely applicable nonparametric entropy-based testing procedure that can be used to assess the validity of simple hypotheses about a specific parametric population distribution. The testing…

Econometrics · Economics 2022-01-19 Ron Mittelhammer , George Judge , Miguel Henry

We present an information-theoretic framework for understanding overfitting and underfitting in machine learning and prove the formal undecidability of determining whether an arbitrary classification algorithm will overfit a dataset.…

Machine Learning · Computer Science 2020-11-10 Daniel Bashir , George D. Montanez , Sonia Sehra , Pedro Sandoval Segura , Julius Lauw

We consider the problem of multiple hypothesis testing with generic side information: for each hypothesis $H_i$ we observe both a p-value $p_i$ and some predictor $x_i$ encoding contextual information about the hypothesis. For large-scale…

Methodology · Statistics 2018-07-26 Lihua Lei , William Fithian