Related papers: From Blackwell Dominance in Large Samples to Renyi…
This paper presents a novel theoretical study of the general problem of multiple source adaptation using the notion of Renyi divergence. Our results build on our previous work [12], but significantly broaden the scope of that work in…
R\'enyi divergence is related to R\'enyi entropy much like information divergence (also called Kullback-Leibler divergence or relative entropy) is related to Shannon's entropy, and comes up in many settings. It was introduced by R\'enyi as…
This paper studies the complexity of estimating Renyi divergences of discrete distributions: $p$ observed from samples and the baseline distribution $q$ known \emph{a priori}. Extending the results of Acharya et al. (SODA'15) on estimating…
One possibility of defining a quantum R\'enyi $\alpha$-divergence of two quantum states is to optimize the classical R\'enyi $\alpha$-divergence of their post-measurement probability distributions over all possible measurements (measured…
We revisit the problem of asymmetric binary hypothesis testing against a composite alternative hypothesis. We introduce a general framework to treat such problems when the alternative hypothesis adheres to certain axioms. In this case we…
We study the link between Phelps-Aigner-Cain-type statistical discrimination and familiar notions of statistical informativeness. Our central insight is that Blackwell's Theorem, suitably relabeled, characterizes statistical discrimination…
I establish a translation invariance property of the Blackwell order over experiments, show that garbling experiments bring them closer together, and use these facts to define a cardinal measure of informativeness. Experiment $A$ is…
There is a well-known problem in Null Hypothesis Significance Testing: many statistically significant results fail to replicate in subsequent experiments. We show that this problem arises because standard `point-form null' significance…
Reproducibility is an important feature of science; experiments are retested, and analyses are repeated. Trust in the findings increases when consistent results are achieved. Despite the importance of reproducibility, significant work is…
In the present follow-up article of a previous one [1] we illustrate the use of the Unconventional Statistical Mechanics described and discussed in the latter. This is done via the analysis, resorting to Renyi approach, of experimental…
Data scarcity drives the need for more sample-efficient large language models. In this work, we use the double descent phenomenon to holistically compare the sample efficiency of discrete diffusion and autoregressive models. We show that…
The weak law of large numbers implies that, under mild assumptions on the source, the Renyi entropy per produced symbol converges (in probability) towards the Shannon entropy rate. This paper quantifies the speed of this convergence for…
Contrastive representation learning seeks to acquire useful representations by estimating the shared information between multiple views of data. Here, the choice of data augmentation is sensitive to the quality of learned representations:…
We propose a natural relaxation of differential privacy based on the Renyi divergence. Closely related notions have appeared in several recent papers that analyzed composition of differentially private mechanisms. We argue that the useful…
Differential privacy is a de facto standard in data privacy, with applications in the public and private sectors. A way to explain differential privacy, which is particularly appealing to statistician and social scientists is by means of…
The achievable error-exponent pairs for the type I and type II errors are characterized in a hypothesis testing setup where the observation consists of independent and identically distributed samples from either a known joint probability…
Renyi's information measures ---the Renyi information, mean, capacity, radius, and center--- are analyzed relying on the elementary properties of the Renyi divergence and the power means. The van Erven-Harremoes conjecture is proved for any…
Real-life data are often non-IID due to complex distributions and interactions, and the sensitivity to the distribution of samples can differ among learning models. Accordingly, a key question for any supervised or unsupervised model is…
We discuss some properties of the generalized entropies, called Renyi entropies and their application to the case of continuous distributions. In particular it is shown that these measures of complexity can be divergent, however, their…
A refinement of the multinomial distribution is presented where the number of inversions in the sequence of outcomes is tallied. This refinement of the multinomial distribution is its joint distribution with the number of inversions in the…