Related papers: On the entropy flows to disorder
Exponential families are widely used in machine learning, including many distributions in continuous and discrete domains (e.g., Gaussian, Dirichlet, Poisson, and categorical distributions via the softmax transformation). Distributions in…
It has been shown that sufficiently well mixing dynamical systems with positive entropy have extreme value laws which in the limit converge to one of the three standard distributions known for i.i.d. processes, namely Gumbel, Fr\'echet and…
The specific relative entropy, introduced by N. Gantert, allows to quantify the discrepancy between the laws of potentially mutually singular measures. It appears naturally as the large deviations rate function in a randomized version of…
Shannon entropy was defined for probability distributions and then its using was expanded to measure the uncertainty of knowledge for systems with complete information. In this article, it is proposed to extend the using of Shannon entropy…
There are given characterizations of the exponential distribution by the properties of the independence of linear forms with random coefficients. Related results based on the constancy of regression of one statistic on a linear form are…
Many partially-successful attempts have been made to find the most natural discrete-variable version of Shannon's entropy power inequality (EPI). We develop an axiomatic framework from which we deduce the natural form of a discrete-variable…
The increased uncertainty and complexity of nonlinear systems have motivated investigators to consider generalized approaches to defining an entropy function. New insights are achieved by defining the average uncertainty in the probability…
The article presents a novel variational calculus to analyze the stability and the propagation of chaos properties of nonlinear and interacting diffusions. This differential methodology combines gradient flow estimates with backward…
We discuss algorithms for estimating the Shannon entropy h of finite symbol sequences with long range correlations. In particular, we consider algorithms which estimate h from the code lengths produced by some compression algorithm. Our…
We study the behavior of a nonlinear semiclassical system using Shannon entropy and two approaches to statistical complexity. These systems involve the interaction between classical variables (representing the environment) and quantum ones.…
A criterion for proving a strong form of propagation of chaos on the path space, known as entropy chaos, for a general interacting diffusion system is proposed. Our analysis focuses on the class of conservative diffusions introduced by…
Using the works of Ma\~n\'e \cite{Ma} and Paternain \cite{Pat} we study the distribution of geodesic arcs with respect to equilibrium states of the geodesic flow on a closed manifold, equipped with a $\mathcal{C}^{\infty}$ Riemannian…
The time variation of entropy, as an alternative to the variance, is proposed as a measure of the diffusion rate. It is shown that for linear and time-translationally invariant systems having a large-time limit for the density, at large…
Claude Shannon coined entropy to quantify the uncertainty of a random distribution for communication coding theory. We observe that the uncertainty nature of entropy also limits its direct usage in mathematical modeling. Therefore we…
We determine the joint limiting distribution of adjacent spacings around a central, intermediate, or an extreme order statistic $X_{k:n}$ of a random sample of size $n$ from a continuous distribution $F$. For central and intermediate cases,…
The Sharma-Mittal entropies generalize the celebrated Shannon, R\'enyi and Tsallis entropies. We report a closed-form formula for the Sharma-Mittal entropies and relative entropies for arbitrary exponential family distributions. We…
A simple expression for the non-equilibrium distribution function in ultra-fast transient processes is proposed. Postulating its dependence on temporal derivatives of the equilibrium integrals of motion, non-equilibrium analogues of the…
We compute analytically the probability distribution function ${\cal P}(\epsilon)$ of the dissipation field $\epsilon =(\nabla \theta)^{2}$ of a passive scalar $\theta$ advected by a $d$-dimensional random flow, in the limit of large Peclet…
A message of any sort can be regarded as a source of information. Claude. E. Shannon showed in the last century that information ("what we don't already know") is equivalent to the entropy as defined in statistical mechanics. A string of…
In this paper we introduce a new method to add a parameter to a family of distributions. The additional parameter is completely studied and a full description of its behaviour in the distribution is given. We obtain several mathematical…