Related papers: Sharp Concentration Inequalities for the Centered …
In this paper, we establish an almost sure central limit theorem for a general random sequence under a strong approximation condition. Additionally, we derive the law of the iterated logarithm for the center of mass corresponding to a…
The quantum relative entropy is a fundamental quantity in quantum information science, characterizing the distinguishability between two quantum states. However, this quantity is not additive in general for correlated quantum states,…
We give a new type of sufficient condition for the existence of measures with maximal entropy for an interval map $f$, using some non-uniform hyperbolicity to compensate for a lack of smoothness of $f$. More precisely, if the topological…
Relative entropy is a fundamental class of distances between probability distributions, with widespread applications in probability theory, statistics, and machine learning. In this work, we study relative entropy from a categorical…
From the output produced by a memoryless deletion channel from a uniformly random input of known length $n$, one obtains a posterior distribution on the channel input. The difference between the Shannon entropy of this distribution and that…
It is well known that, under standard regularity conditions, the maximum likelihood estimator (MLE) satisfies a central limit theorem and converges in distribution to a Gaussian random variable as the sample size grows. This paper…
Compact data representations are one approach for improving generalization of learned functions. We explicitly illustrate the relationship between entropy and cardinality, both measures of compactness, including how gradient descent on the…
New Vapnik and Chervonenkis type concentration inequalities are derived for the empirical distribution of an independent random sample. Focus is on the maximal deviation over classes of Borel sets within a low probability region. The…
Matrix concentration inequalities provide a direct way to bound the typical spectral norm of a random matrix. The methods for establishing these results often parallel classical arguments, such as the Laplace transform method. This work…
We study the problem of identifying the causal relationship between two discrete random variables from observational data. We recently proposed a novel framework called entropic causality that works in a very general functional model but…
Recent works in dimensionality reduction for regression tasks have introduced the notion of sensitivity, an estimate of the importance of a specific datapoint in a dataset, offering provable guarantees on the quality of the approximation…
We show how to determine the maximum and minimum possible values of one measure of entropy for a given value of another measure of entropy. These maximum and minimum values are obtained for two standard forms of probability distribution (or…
We study the statistical properties of the entropic optimal (self) transport problem for smooth probability measures. We provide an accurate description of the limit distribution for entropic (self-)potentials and plans as the…
In classical and quantum information theory, operational quantities such as the amount of randomness that can be extracted from a given source or the amount of space needed to store given data are normally characterized by one of two…
We prove for the rescaled convolution map $f\to f\circledast f$ propagation of polynomial, exponential and gaussian localization. The gaussian localization is then used to prove an optimal bound on the rate of entropy production by this…
In this paper we derive the maximum entropy characteristics of a particular rank order distribution, namely the discrete generalized beta distribution, which has recently been observed to be extremely useful in modelling many several…
Given two discrete random variables $X$ and $Y$, with probability distributions ${\bf p} =(p_1, \ldots , p_n)$ and ${\bf q}=(q_1, \ldots , q_m)$, respectively, denote by ${\cal C}({\bf p}, {\bf q})$ the set of all couplings of ${\bf p}$ and…
Within the setting of rare event modelling, the method of level sets allows us to define an equivalence relation over rare events with distinct rates of entropy production. This method allows us to clarify the relation between the empirical…
A new framework is introduced for examining and evaluating the fundamental limits of lossless data compression, that emphasizes genuinely non-asymptotic results. The {\em sample complexity} of compressing a given source is defined as the…
This dissertation investigates relative entropies, also called generalized divergences, and how they can be used to characterize information-theoretic tasks in quantum information theory. The main goal is to further refine characterizations…