相关论文: Empirical entropy in context

Relations between Shannon entropy and Renyi entropies of integer order are discussed. For any N-point discrete probability distribution for which the Renyi entropies of order two and three are known, we provide an lower and an upper bound…

The new estimates of the conditional Shannon entropy are introduced in the framework of the model describing a discrete response variable depending on a vector of d factors having a density w.r.t. the Lebesgue measure in R^d. Namely, the…

We consider a strictly substochastic matrix or an stochastic matrix with absorbing states. By using quasi-stationary distributions one shows there is a canonical associated stationary Markov chain. Based upon $2-$stringing representation of…

Entropic uncertainty relations are interesting in their own rights as well as for a lot of applications. Keeping this in mind, we try to make the corresponding inequalities as tight as possible. The use of parametrized entropies also allows…

We perform a throughout numerical study of the average sensitivity to initial conditions and entropy production for two symplectically coupled standard maps focusing on the control-parameter region close to regularity. Although the system…

Markov categories are a novel framework to describe and treat problems in probability and information theory. In this work we combine the categorical formalism with the traditional quantitative notions of entropy, mutual information, and…

We introduce and study the notion of a directional complexity and entropy for maps of degree 1 on the circle. For piecewise affine Markov maps we use symbolic dynamics to relate this complexity to the symbolic complexity. We apply a…

The aim of the paper is to study the link between non additivity of some entropies and their boundedness. We propose an axiomatic construction of the entropy relying on the fact that entropy belongs to a group isomorphic to the usual…

Shannon's entropy and other entropy-based concepts are derived from the new, more general concept of relative divergence of one "grading' function on a linearly ordered set from another such function. The definition of relative divergence…

Entropy production is a key quantity in any finite-time thermodynamic process. It is intimately tied with the fundamental laws of thermodynamics, embodying a tool to extend thermodynamic considerations all the way to non-equilibrium…

With the rapid expansion of graphs and networks and the growing magnitude of data from all areas of science, effective treatment and compression schemes of context-dependent data is extremely desirable. A particularly interesting direction…

This note provides several recent progresses in the study of long time behavior of Markov processes. The examples presented below are related to other scientific fields as PDE's, physics or biology. The involved mathematical tools as…

This paper introduces an objective metric for evaluating a parsing scheme. It is based on Shannon's original work with letter sequences, which can be extended to part-of-speech tag sequences. It is shown that this regular language is an…

Ever since Claude Shannon used entropy for his "Mathematical Theory of Communication", entropy has become a buzzword in research circles with scientists applying entropy to describe any phenomena that are reminiscent of disorder. In this…

The LMC complexity, an indicator of complexity based on a probabilistic description, is revisited. A straightforward approach allows us to establish the time evolution of this indicator in a near-equilibrium situation and gives us a new…

We apply a common measure of randomness, the entropy, in the context of iterated functions on a finite set with n elements. For a permutation, it turns out that this entropy is asymptotically (for a growing number of iterations) close to…

In this paper, I expand Shannon's definition of entropy into a new form of entropy that allows integration of information from different random events. Shannon's notion of entropy is a special case of my more general definition of entropy.…

The major problem in information theoretic analysis of neural responses and other biological data is the reliable estimation of entropy--like quantities from small samples. We apply a recently introduced Bayesian entropy estimator to…

Shannon entropy, a cornerstone of information theory, statistical physics and inference methods, is uniquely identified by the Shannon-Khinchin or Shore-Johnson axioms. Generalizations of Shannon entropy, motivated by the study of…

The principle of entropy increase is not only the basis of statistical mechanics, but also closely related to the irreversibility of time, the origin of life, chaos and turbulence. In this paper, we first discuss the dynamic system…