Related papers: Metric entropy for functions of bounded total gene…
Minimum divergence estimators provide a natural choice of estimators in a statistical inference problem. Different properties of various families of these divergence measures such as Hellinger distance, power divergence, density power…
Finite mixture models have long been used across a variety of fields in engineering and sciences. Recently there has been a great deal of interest in quantifying the convergence behavior of the \emph{mixing measure}, a fundamental object…
Behavior of the entropy numbers of classes of multivariate functions with mixed smoothness is studied here. This problem has a long history and some fundamental problems in the area are still open. The main goal of this paper is to develop…
The entropy is a measure of uncertainty that plays a central role in information theory. When the distribution of the data is unknown, an estimate of the entropy needs be obtained from the data sample itself. We propose a semi-parametric…
In this paper we develop a theory of matrix completion for the extreme case of noisy 1-bit observations. Instead of observing a subset of the real-valued entries of a matrix M, we obtain a small number of binary (1-bit) measurements…
We describe a method to computationally estimate the probability density function of a univariate random variable by applying the maximum entropy principle with some local conditions given by Gaussian functions. The estimation errors and…
Entropy rate is a real valued functional on the space of discrete random sources which lacks a closed formula even for subclasses of sources which have intuitive parameterizations. A good way to overcome this problem is to examine its…
We present a new methodology for the characterization of the metric entropy of infinite-dimensional ellipsoids with exponentially decaying semi-axes. This procedure does not rely on the explicit construction of coverings or packings and…
We revisit generalized entropic formulations of the uncertainty principle for an arbitrary pair of quantum observables in two-dimensional Hilbert space. R\'enyi entropy is used as uncertainty measure associated with the distribution…
We establish the first information-theoretic limits for multimodal retrieval. Casting ranking as lossy source coding, we derive a single-letter rate-distortion function $R(D)$ for reciprocal-rank distortion and prove a converse bound that…
A novel, non-trivial, probabilistic upper bound on the entropy of an unknown one-dimensional distribution, given the support of the distribution and a sample from that distribution, is presented. No knowledge beyond the support of the…
For \Gamma a countable amenable group consider those actions of \Gamma as measure-preserving transformations of a standard probability space, written as {T_\gamma}_{\gamma \in \Gamma} acting on (X,{\cal F}, \mu). We say…
Estimating the entropy rate of discrete time series is a challenging problem with important applications in numerous areas including neuroscience, genomics, image processing and natural language processing. A number of approaches have been…
Given a probability distribution P, what is the minimum amount of bits needed to store a value x sampled according to P, such that x can later be recovered (except with some small probability)? Or, what is the maximum amount of uniform…
Entropy estimation is of practical importance in information theory and statistical science. Many existing entropy estimators suffer from fast growing estimation bias with respect to dimensionality, rendering them unsuitable for…
A novel principle is presented which allows for the proof of bounded weak solutions to a class of physically relevant, strongly coupled parabolic systems exhibiting a formal gradient-flow structure. The main feature of these systems is that…
We seek an entropy estimator for discrete distributions with fully empirical accuracy bounds. As stated, this goal is infeasible without some prior assumptions on the distribution. We discover that a certain information moment assumption…
Simple upper and lower bounds are obtained for the integral $\int_0^x\mathrm{e}^{-\gamma t}t^\nu I_\nu(t)\,\mathrm{d}t$, $x>0$, $\nu>-\frac{1}{2}$, $0<\gamma<1$. Most of our bounds for this integral are tight as $x\rightarrow\infty$. We…
We consider the problem of learning a target probability distribution over a set of $N$ binary variables from the knowledge of the expectation values (with this target distribution) of $M$ observables, drawn uniformly at random. The space…
We introduce a new distance-preserving compact representation of multi-dimensional point-sets. Given $n$ points in a $d$-dimensional space where each coordinate is represented using $B$ bits (i.e., $dB$ bits per point), it produces a…