Related papers: Necessary and Sufficient Conditions for Characteri…

This paper develops a new framework for indirect statistical inference with guaranteed necessity and sufficiency, applicable to continuous random variables. We prove that when comparing exponentially transformed order statistics from an…

It was recently shown that estimating the Shannon entropy $H({\rm p})$ of a discrete $k$-symbol distribution ${\rm p}$ requires $\Theta(k/\log k)$ samples, a number that grows near-linearly in the support size. In many applications $H({\rm…

Permutation Entropy, introduced by Bandt and Pompe, is a widely used complexity measure for real-valued time series that is based on the relative order of values within consecutive segments of fixed length. After standardizing each segment…

We derive a sufficient condition for a $k$-th order homogeneous Markov chain $\mathbf{Z}$ with finite alphabet $\mathcal{Z}$ to have a unique invariant distribution on $\mathcal{Z}^k$. Specifically, let $\mathbf{X}$ be a first-order,…

Convergence properties of Shannon Entropy are studied. In the differential setting, it is shown that weak convergence of probability measures, or convergence in distribution, is not enough for convergence of the associated differential…

We prove a Bernstein-type bound for the difference between the average of negative log-likelihoods of independent discrete random variables and the Shannon entropy, both defined on a countably infinite alphabet. The result holds for the…

Shannon entropy for discrete distributions is a fundamental and widely used concept, but its continuous analogue, known as differential entropy, lacks essential properties such as positivity and compatibility with the discrete case. In this…

We provide necessary and sufficient conditions for a tempered distribution $F\in S'(R)$ to be positive definite. A generalized Cauchy transform $\widetilde{F}$ of $F$ is used as a numerical continuation of $F$ to the open upper and lower…

A class of discrete probability distributions contains distributions with limited support. A typical example is some variant of a Likert scale, with response mapped to either the $\{1, 2, \ldots, 5\}$ or $\{-3, -2, \ldots, 2, 3\}$ set. An…

The problem of Shannon entropy estimation in countable infinite alphabets is addressed from the study and use of convergence results of the entropy functional, which is known to be discontinuous with respect to the total variation distance…

Even if a probability distribution is properly normalizable, its associated Shannon (or von Neumann) entropy can easily be infinite. We carefully analyze conditions under which this phenomenon can occur. Roughly speaking, this happens when…

A new characterization of the exponential distribution is established. It is proven that the well-known Sukhatme-Renyi necessary condition is also sufficient for exponentiality. A method of proof due to Arnold and Villasenor based on the…

Let $ X_1, \ldots, X_n $ be independent random variables taking values in the alphabet $ \{0, 1, \ldots, r\} $, and $ S_n = \sum_{i = 1}^n X_i $. The Shepp--Olkin theorem states that, in the binary case ($ r = 1 $), the Shannon entropy of $…

In this paper we focus on providing sufficient conditions for some well-known stochastic orders in reliability but dealing with the discrete versions of them, filling a gap in the literature. In particular, we find conditions based on the…

In this paper, we provide the proof of $L^2$ consistency for the $k$th nearest neighbour distance estimator of the Shannon entropy for an arbitrary fixed $k\geq 1.$ We construct the non-parametric test of goodness-of-fit for a class of…

We study the independence structure of finitely exchangeable distributions over random vectors and random networks. In particular, we provide necessary and sufficient conditions for an exchangeable vector so that its elements are completely…

Let $G$ be a connected semisimple simply connected Lie group with a compact Cartan subgroup and let $\Gamma$ be a uniform lattice in $G$. Let $\widehat{G}_d$ denote the set of equivalence classes of unitary discrete series representations…

Given a countable sofic group $\Gamma$, a finite alphabet $A$, a subshift $X \subseteq A^\Gamma$, and a potential $\phi: X \to \mathbb{R}$, we give sufficient conditions on $X$ and $\phi$ for expressing, in the uniqueness regime, the sofic…

We have presented a new axiomatic derivation of Shannon Entropy for a discrete probability distribution on the basis of the postulates of additivity and concavity of the entropy function.We have then modified shannon entropy to take account…

The classical problem of maximizing the Shannon entropy of a sum of independent random variables supported on a finite alphabet is considered and settled in the ternary case. Namely, the following theorem is established: if…