Related papers: Three tutorial lectures on entropy and counting
This paper describes an entropy equation, but one that should be used for measuring energy and not information. In relation to the human brain therefore, both of these quantities can be used to represent the stored information. The human…
Entropy is a central concept in physics, but can be challenging to calculate even for systems that are easily simulated. This is exacerbated out of equilibrium, where generally little is known about the distribution characterizing simulated…
Typically, the entropy of an isolated system in equilibrium is calculated by counting the number of accessible microstates, or in more general cases by using the Gibbs formula. In irreversible processes entropy spontaneously increases and…
This paper addresses the challenge of identifying a minimal subset of discrete, independent variables that best predicts a binary class. We propose an efficient iterative method that sequentially selects variables based on which one…
These notes provide a brief primer on the basic aspects of "observational entropy" (also known as "quantum coarse-grained entropy"), a general framework for applying the concept of coarse-graining to quantum systems. We review the basic…
In this article we discuss a model used to introduce the concept of entropy with secondary school students. It can be used to discuss with students the reversibility of time, the tendency towards homogeneity and the link between probability…
Certified randomness guaranteed to be unpredictable by adversaries is central to information security. The fundamental randomness inherent in quantum physics makes certification possible from devices that are only weakly characterised, i.e.…
We study entropy production (EP) in processes involving repeated quantum measurements of finite quantum systems. Adopting a dynamical system approach, we develop a thermodynamic formalism for the EP and study fine aspects of irreversibility…
This letter reports two moment extensions of the entropy of a distribution. By understanding the traditional entropy as the average of the original distribution up to a random variable transformation, the traditional moments equation become…
Entropy is a quantity for counting physical degrees of freedom in a system. At a finite temperature, one can use thermal entropy to study thermodynamical properties. At zero temperature, entanglement entropy is expected to provide a…
We study the sequence entropy for amenable group actions and investigate systematically spectrum and several mixing concepts via sequence entropy both in measure-theoretic dynamical systems and topological dynamical systems. Moreover, we…
Problems of probabilistic inference and decision making under uncertainty commonly involve continuous random variables. Often these are discretized to a few points, to simplify assessments and computations. An alternative approximation is…
We examine the statistical number of states, from which statistical entropy can be derived, and we show that it is an explicit function of the metric and thus observer dependent. We find a constraint on a transformation of the metric that…
We consider distributions of ordered random vectors with given one-dimensional marginal distributions. We give an elementary necessary and sufficient condition for the existence of such a distribution with finite entropy. In this case, we…
The interactions between three or more random variables are often nontrivial, poorly understood, and yet, are paramount for future advances in fields such as network information theory, neuroscience, genetics and many others. In this work,…
We consider the entropy of sums of independent discrete random variables, in analogy with Shannon's Entropy Power Inequality, where equality holds for normals. In our case, infinite divisibility suggests that equality should hold for…
For control systems in discrete time, this paper discusses measure-theoretic invariance entropy for a subset Q of the state space with respect to a quasi-stationary measure obtained by endowing the control range with a probability measure.…
Entropy Estimation is an important problem with many applications in cryptography, statistic,machine learning. Although the estimators optimal with respect to the sample complexity have beenrecently developed, there are still some…
Logical entropy gives a measure, in the sense of measure theory, of the distinctions of a given partition of a set, an idea that can be naturally generalized to classical probability distributions. Here, we analyze how fundamental concepts…
A pedagogical account of entropy and its history.