Related papers: Concavity of entropy under thinning

Let $0 < \epsilon < 1/2$ be a noise parameter, and let $T_{\epsilon}$ be the noise operator acting on functions on the boolean cube $\{0,1\}^n$. Let $f$ be a nonnegative function on $\{0,1\}^n$. We upper bound the entropy of $T_{\epsilon}…

Shannon entropy in position ($S_{\rvec}$) and momentum ($S_{\pvec}$) spaces, along with their sum ($S_t$) are presented for unit-normalized densities of He, Li$^+$ and Be$^{2+}$ ions, spatially confined at the center of an impenetrable…

This paper considers the estimation of Shannon entropy for discrete distributions with countably infinite support. While minimax rates for finite-support distributions are established, infinite-support distributions present distinct…

We show that under natural growth conditions on the entropy function, convergence in relative entropy is equivalent to $L_p$-convergence. The main tool is the theory of Young measures, in a form that accounts for the formation of…

We consider a pair of correlated processes {Z_n} and {S_n} (two sided), where the former is observable and the later is hidden. The uncertainty in the estimation of Z_n upon its finite past history is H(Z_n|Z_0^{n-1}), and for estimation of…

Motivated by connections to random matrices, Littlewood-Richardson coefficients and tilings, we study random discrete concave functions on an equilateral lattice. We show that such functions having a periodic Hessian of a fixed average…

Let $f$ be a $C^r$ ($r>1$) diffeomorphism on a compact surface $M$ with $h_{\rm top}(f)\geq\frac{\lambda^{+}(f)}{r}$ where $\lambda^{+}(f):=\lim_{n\to+\infty}\frac{1}{n}\max_{x\in M}\log \left\|Df^{n}_{x}\right\|$. We establish an…

We show that a proper expression of the uncertainty relation for a pair of canonically-conjugate continuous variables relies on entropy power, a standard notion in Shannon information theory for real-valued signals. The resulting…

The elliptic integral and its various generalizations are playing very important and basic role in different branches of modern mathematics. It is well known that they cannot be represented by the elementary transcendental functions.…

We study an entropy measure for quantum systems that generalizes the von Neumann entropy as well as its classical counterpart, the Gibbs or Shannon entropy. The entropy measure is based on hypothesis testing and has an elegant formulation…

There are numerous characterizations of Shannon entropy and Tsallis entropy as measures of information obeying certain properties. Using work by Faddeev and Furuichi, we derive a very simple characterization. Instead of focusing on the…

Given a suitably normalized $X\in\mathbb{R}^n$ we observe that the function $\theta\mapsto\mathbb{E}|X\cdot\theta|$, defined for $\theta\in S^{n-1}$, admits surprisingly strong concentration far surpassing what is expected on account of…

For $X$, $Y$, $Z$ and $W$ compact metric spaces, consider two uniformly contractive IFS $\{\tau_x: Z\to Z,\, x\in x\}$ and $\{\tau_y:W\to W,\, y\in Y\}$. For a fixed $\alpha \in \mathcal{P}(X)$ with $supp(\alpha)=X$ we define the entropy of…

For a subshift $(X, \sigma_X)$ and a subadditive sequence $\mathcal{F}=\{\log f_n\}_{n=1}^{\infty}$ on $X$, we study equivalent conditions for the existence of $h\in C(X)$ such that $\lim_{n\rightarrow\infty}(1/{n})\int \log f_n d \mu=\int…

Uncertainty relations are central to quantum physics. While they were originally formulated in terms of variances, they have later been successfully expressed with entropies following the advent of Shannon information theory. Here, we…

Mixed volumes, which are the polarization of volume with respect to the Minkowski addition, are fundamental objects in convexity. In this note we announce the construction of mixed integrals, which are functional analogs of mixed volumes.…

The Shannon entropy of a random variable has much behaviour analogous to a signed measure. Previous work has explored this connection by defining a signed measure on abstract sets, which are taken to represent the information that different…

This paper develops a general theory on rates of convergence of penalized spline estimators for function estimation when the likelihood functional is concave in candidate functions, where the likelihood is interpreted in a broad sense that…

The paper describes an approach to measuring convergence of an algorithm to its result in terms of an entropy-like function of partitions of its inputs of a given length. The goal is to look at the algorithmic data processing from the…

We introduce a generalization of relative entropy derived from the Wigner-Yanase-Dyson entropy and give a simple, self-contained proof that it is convex. Moreover, special cases yield the joint convexity of relative entropy, and for the map…