Related papers: Statistical Estimation of Conditional Shannon Entr…
The uncertainty relation for continuous variables due to Byalinicki-Birula and Mycielski expresses the complementarity between two $n$-uples of canonically conjugate variables $(x_1,x_2,\cdots x_n)$ and $(p_1,p_2,\cdots p_n)$ in terms of…
In recent decades, several definitions of new entropy measures have been proposed, which expands the range of applications for this important tool. The present work focuses on the extension of the classical Shannon entropy to the hyperbolic…
We show that the way in which the Shannon entropy of sequences produced by an information source converges to the source's entropy rate can be used to monitor how an intelligent agent builds and effectively uses a predictive model of its…
A new combinatorial-probabilistic diagnostic entropy has been introduced. It describes the pair-wise sum of probabilities of system conditions that have to be distinguished during the diagnosing process. The proposed measure describes the…
A new information theoretic condition is presented for reconstructing a discrete random variable $X$ based on the knowledge of a set of discrete functions of $X$. The reconstruction condition is derived from Shannon's 1953 lattice theory…
A probabilistic source is defined as the set of infinite words (over a given denumerable alphabet) endowed with a probability $\mu$. The paper deals with general binary sources where the distribution of any symbol (0 or 1) may depend on an…
We develop Bayesian models for density regression with emphasis on discrete outcomes. The problem of density regression is approached by considering methods for multivariate density estimation of mixed scale variables, and obtaining…
A general method is presented for modeling high entropy alloys as ensembles of randomly sampled, ordered configurations on a given lattice. Statistical mechanics is applied post hoc to derive the ensemble properties as a function of…
Entropies must correspond to mean values for them to be measurable. The Shannon entropy corresponds to the weighted arithmetic mean, whereas the Renyi entropy corresponds to the exponential mean. These means refer to code lengths, which are…
This paper defines and discusses the dimension notion of topological slow entropy of any subset for Z^d actions. Also, the notion of measure-theoretic slow entropy for Z^d actions is presented, which is modified from Brin and Katok [2].…
This paper proposes a new method for estimating the joint probability mass function of a pair of discrete random variables. This estimator is used to construct joint Shannon R\'enyi-Tsallis entropies, and the mutual information estimates of…
R\'enyi entropy is a one-parameter generalization of Shannon entropy, which has been used in various fields of physics. Despite its wide applicability, the physical interpretations of the R\'enyi entropy are not widely known. In this paper,…
We consider the problem of approximating the empirical Shannon entropy of a high-frequency data stream under the relaxed strict-turnstile model, when space limitations make exact computation infeasible. An equivalent measure of entropy is…
In this paper, we provide the proof of $L^2$ consistency for the $k$th nearest neighbour distance estimator of the Shannon entropy for an arbitrary fixed $k\geq 1.$ We construct the non-parametric test of goodness-of-fit for a class of…
I consider the effect of a finite sample size on the entropy of a sample of independent events. I propose formula for entropy which satisfies Shannon's axioms, and which reduces to Shannon's entropy when sample size is infinite. I discuss…
The uncertainty relation based on the Shannon entropies of the probability densities in position and momentum spaces is improved for quantum systems in arbitrary $D$-dimensional spherically symmetric potentials. To find it, we have used the…
Observing how long a dynamical system takes to return to some state is one of the most simple ways to model and quantify its dynamics from data series. This work proposes two formulas to estimate the KS entropy and a lower bound of it, a…
The problems of causality, modeling, and control for chaotic, high-dimensional dynamical systems are formulated in the language of information theory. The central quantity of interest is the Shannon entropy, which measures the amount of…
Product probability property, known in the literature as statistical independence, is examined first. Then generalized entropies are introduced, all of which give generalizations to Shannon entropy. It is shown that the nature of the…
Estimates of densities of convolution semigroups of probability measures are given under specific assumptions on the corresponding L\'evy measure and the L\'evy--Khinchin exponent. The assumptions are satisfied, e.g., by tempered stable…