Related papers: Fractional differential entropy and its applicatio…
The Shannon entropy, and related quantities such as mutual information, can be used to quantify uncertainty and relevance. However, in practice, it can be difficult to compute these quantities for arbitrary probability distributions,…
The present paper aims to investigate the metric mean dimension theory of continuous flows. We introduce the notion of metric mean dimension for continuous flows to characterize the complexity of flows with infinite topological entropy. For…
We calculate and analyze various entropy measures and their properties for selected probability distributions. The entropies considered include Shannon, R\'enyi, generalized R\'enyi, Tsallis, Sharma-Mittal, and modified Shannon entropy,…
Following the work on Shannon entropy together with the principle of maximum entropy, Luo & Singh (J. Hydrol. Eng., 2011, 16(4): 303-315) and Singh & Luo (J. Hydrol. Eng., 2011, 16(9): 725-735) explored the concept of non-extensive Tsallis…
We present a new class of estimators of Shannon entropy for severely undersampled discrete distributions. It is based on a generalization of an estimator proposed by T. Schuermann, which itself is a generalization of an estimator proposed…
This paper investigates what can be inferred about an arbitrary continuous probability distribution from a finite sample of $N$ observations drawn from it. The central finding is that the $N$ sorted sample points partition the real line…
In this paper, we develop the notion of entropy for uniform hypergraphs via tensor theory. We employ the probability distribution of the generalized singular values, calculated from the higher-order singular value decomposition of the…
We analyze dynamical properties of the Shannon information entropy of a continuous probability distribution, which is driven by a standard diffusion process. This entropy choice is confronted with another option, employing the conditional…
We present a detailed derivation of some estimators of Shannon entropy for discrete distributions. They hold for finite samples of N points distributed into M "boxes", with N and M -> oo, but N/M < oo. In the high sampling regime (<< 1…
The time variation of entropy, as an alternative to the variance, is proposed as a measure of the diffusion rate. It is shown that for linear and time-translationally invariant systems having a large-time limit for the density, at large…
We investigate the efficiency of several types of continued fraction expansions of a number in the unit interval using a generalization of Lochs theorem from 1964. Thus, we aimed to compare the efficiency by describing the rate at which the…
In one-dimensional diffusive processes with discrete steps characterized by geometrically decaying magnitudes, the usual Gaussian broadening familiar from Brownian motion is replaced by bounded probability distributions over particle…
Fractional derivatives can be used to model time delays in a diffusion process. When the order of the fractional derivative is distributed over the unit interval, it is useful for modeling a mixture of delay sources. In some special cases…
We introduce some new classes of unimodal rotational invariant directional distributions, which generalize von Mises-Fisher distribution. We propose three types of distributions, one of which represents axial data. For each new type we…
The problem of Shannon entropy estimation in countable infinite alphabets is addressed from the study and use of convergence results of the entropy functional, which is known to be discontinuous with respect to the total variation distance…
The statistical analysis of data stemming from dynamical systems, including, but not limited to, time series, routinely relies on the estimation of information theoretical quantities, most notably Shannon entropy. To this purpose, possibly…
Entropy estimation is of practical importance in information theory and statistical science. Many existing entropy estimators suffer from fast growing estimation bias with respect to dimensionality, rendering them unsuitable for…
Convergence properties of Shannon Entropy are studied. In the differential setting, it is shown that weak convergence of probability measures, or convergence in distribution, is not enough for convergence of the associated differential…
Entropy and differential entropy are important quantities in information theory. A tractable extension to singular random variables-which are neither discrete nor continuous-has not been available so far. Here, we present such an extension…
A theoretical formula for the probability density function (PDF) of velocity derivatives in a fully developed turbulent flow is derived with the multifractal aspect based on the generalized measures of entropy, i.e., the extensive Renyi…