Related papers: Three tutorial lectures on entropy and counting
In this article, I give a definition of topological entropy for random dynamical systems associated to an infinite countable discrete amenable group action. I obtain a variational principle between the topological entropy and measurable…
The entropy of the random walk on the discrete contable group could be used for comparison of the system of the generators. Fundamental inequality between growth, entropy and escape gives the possibility to define "the best" system of the…
For any discrete time dynamical system with a rational evolution, we define an entropy, which is a global index of complexity for the evolution map. We analyze its basic properties and its relations to the singularities and the…
The relationship between three probability distributions and their maximizable entropy forms is discussed without postulating entropy property. For this purpose, the entropy I is defined as a measure of uncertainty of the probability…
In the paper we throw the first light on studying systematically the local entropy theory for a countable discrete amenable group action. For such an action, we introduce entropy tuples in both topological and measure-theoretic settings and…
We present some new nonparametric estimators of entropies and we establish almost sure consistency and central limit Theorems for some of the most important entropies in the discrete case. Our theorical results are validated by simulations.
We explore an asymptotic behavior of entropies for sums of independent random variables that are convolved with a small continuous noise.
Inferring models, predicting the future, and estimating the entropy rate of discrete-time, discrete-event processes is well-worn ground. However, a much broader class of discrete-event processes operates in continuous-time. Here, we provide…
Let $X$ be a discrete random variable with support $S$ and $f : S \to S^\prime$ be a bijection. Then it is well-known that the entropy of $X$ is the same as the entropy of $f(X)$. This entropy preservation property has been well-utilized to…
Combinatorial enumeration leads to counting generating functions presenting a wide variety of analytic types. Properties of generating functions at singularities encode valuable information regarding asymptotic counting and limit…
We develop entropy and variance results for the product of independent identically distributed random variables on Lie groups. Our results apply to the study of stationary measures in various contexts.
Close connections between various notions of entropy and the apparatus of category theory have been observed already in the 1980s and more vigorously developed in the past ten years. The starting point of the paper is the recent categorical…
This text provides a practical introduction to randomness and data analysis, in particular in the context of computer simulations. At the beginning, the most basics concepts of probability are given, in particular discrete and continuous…
Various properties of relative entropy have led to its widespread use in information theory. These properties suggest that relative entropy has a role to play in systems that attempt to perform inference in terms of probability…
We present a general definition of entropy in the setting of pre-ordered semigroups, extending the notion of topological entropy. From our definition, we obtain the basic properties exhibited by various entropy-like theories encountered in…
Entropy is a quantity which is of great importance in physics and chemistry. The concept comes out of thermodynamics, proposed by Rudolf Clausius in his analysis of Carnot cycle and linked by Ludwig Boltzmann to the number of specific ways…
Combinatorics, like computer science, often has to deal with large objects of unspecified (or unusable) structure. One powerful way to deal with such an arbitrary object is to decompose it into more usable components. In particular, it has…
We propose a compression-based version of the empirical entropy of a finite string over a finite alphabet. Whereas previously one considers the naked entropy of (possibly higher order) Markov processes, we consider the sum of the…
Compact data representations are one approach for improving generalization of learned functions. We explicitly illustrate the relationship between entropy and cardinality, both measures of compactness, including how gradient descent on the…
Entropy is a measure of heterogeneity widely used in applied sciences, often when data are collected over space. Recently, a number of approaches has been proposed to include spatial information in entropy. The aim of entropy is to…