Related papers: An Effective Bernstein-type Bound on Shannon Entro…
We prove a general lower bound of quantum decision tree complexity in terms of some entropy notion. We regard the computation as a communication process in which the oracle and the computer exchange several rounds of messages, each round…
The ultimate purpose of the statistical analysis of ordinal patterns is to characterize the distribution of the features they induce. In particular, knowing the joint distribution of the pair Entropy-Statistical Complexity for a large class…
In this paper, we show that the characterisation of all determinant inequalities for $n \times n$ positive definite matrices is equivalent to determining the smallest closed and convex cone containing all entropy functions induced by $n$…
In this paper, the authors provide a weak decoding version of the traditional source coding theorem of Claude Shannon. The central bound that is obtained is \[ \chi>\log_{\epsilon}(2^{-n(H(X)+\epsilon)}) \] where \[…
Grouped data are commonly encountered in applications. The Bernstein polynomial model is proposed as an approximate model in this paper for estimating a univariate density function based on grouped data. The coefficients of the Bernstein…
This paper is devoted to establishing exponential bounds for the probabilities of deviation of a sample sum from its expectation, when the variables involved in the summation are obtained by sampling in a finite population according to a…
Given a mixture between two populations of coins, "positive" coins that each have -- unknown and potentially different -- bias $\geq\frac{1}{2}+\Delta$ and "negative" coins with bias $\leq\frac{1}{2}-\Delta$, we consider the task of…
A state $\rho=(\rho_n)_{n=1}^{\infty}$ is a sequence such that $\rho_n$ is a density matrix on $n$ qubits. It formalizes the notion of an infinite sequence of qubits. The von Neumann entropy $H(d)$ of a density matrix $d$ is the Shannon…
The existing upper and lower bounds between entropy and error probability are mostly derived from the inequality of the entropy relations, which could introduce approximations into the analysis. We derive analytical bounds based on the…
The more then hundred years old Bernstein inequality states that the supremum norm of the derivative of a trigonometric polynomial of fixed degree can be bounded from above by supremum norm of the polynomial itself. The reversed Bernstein…
Recent research has made significant progress on the problem of bounding log partition functions for exponential family graphical models. Such bounds have associated dual parameters that are often used as heuristic estimates of the marginal…
In modern scientific research, small-scale studies with limited participants are increasingly common. However, interpreting individual outcomes can be challenging, making it standard practice to combine data across studies using random…
Relations between Shannon entropy and Renyi entropies of integer order are discussed. For any N-point discrete probability distribution for which the Renyi entropies of order two and three are known, we provide an lower and an upper bound…
Shannon Information theory has achieved great success in not only communication technology where it was originally developed for but also many other science and engineering fields such as machine learning and artificial intelligence.…
Let $ V_{n} = X_{1,n} + X_{2,n} + \cdots + X_{n,n}$ where $X_{i,n}$ are Bernoulli random variables which take the value $1$ with probability $b(i;n)$. Let $\lambda_{n} = \sum\limits_{i=1}^{n} b(i;n) $, $\lambda = \lim\limits_{n \to \infty}…
This paper presents new lower and upper bounds for the optimal compression of binary prefix codes in terms of the most probable input symbol, where compression efficiency is determined by the nonlinear codeword length objective of…
This paper investigates what can be inferred about an arbitrary continuous probability distribution from a finite sample of $N$ observations drawn from it. The central finding is that the $N$ sorted sample points partition the real line…
A method for estimating the Shannon differential entropy of multidimensional random variables using independent samples is described. The method is based on decomposing the distribution into a product of the marginal distributions and the…
While useful probability bounds for $n$ pairwise independent Bernoulli random variables adding up to at least an integer $k$ have been proposed in the literature, none of these bounds are tight in general. In this paper, we provide several…
We consider upper exponential bounds for the probability of the event that an absolute deviation of sample mean from mathematical expectation p is bigger comparing with some ordered level epsilon. These bounds include 2 coefficients {alpha,…