Related papers: Polyadic Entropy, Synergy and Redundancy among Sta…
The study of irreducible higher-order interactions has become a core topic of study in complex systems. Two of the most well-developed frameworks, topological data analysis and multivariate information theory, aim to provide formal tools…
The concept of entropy connects the number of possible configurations with the number of variables in large stochastic systems. Independent or weakly interacting variables render the number of configurations scale exponentially with the…
We investigate the concept of entropy in probabilistic theories more general than quantum mechanics, with particular reference to the notion of information causality recently proposed by Pawlowski et. al. (arXiv:0905.2992). We consider two…
The question of characterization of the degree of non-equilibrium activity in active matter systems is studied in the context of a stochastic microswimmer model driven by a chemical cycle. The resulting dynamical properties and entropy…
Maximum entropy estimation is of broad interest for inferring properties of systems across many different disciplines. In this work, we significantly extend a technique we previously introduced for estimating the maximum entropy of a set of…
Physical systems behave according to their underlying dynamical equations which, in turn, can be identified from experimental data. Explaining data requires selecting mathematical models that best capture the data regularities. Identifying…
Cells sense environmental signals and transmit information intracellularly through changes in the abundance of molecular components. Such molecular abundances can be measured in single cells and exhibit significant heterogeneity in clonal…
Some general considerations on the notion of entropy in physics are presented. An attempt is made to clarify the question of the differentiation between physical entropy (the Clausius-Boltzmann one) and quantities called entropies…
We distinguish a mechanical representation of the world in terms of point masses with positions and momenta and the chemical representation of the world in terms of populations of different individuals, each with intrinsic stochasticity,…
The statistical mechanics of Gibbs is a juxtaposition of subjective, probabilistic ideas on the one hand and objective, mechanical ideas on the other. In this paper, we follow the path set out by Jaynes, including elements added…
Imaging systems are commonly described using resolution, contrast, and signal-to-noise ratio, but these quantities do not provide a general account of how physical transformations affect the flow of information. This paper introduces an…
The concept of entropy in statistical physics is related to the existence of irreversible macroscopic processes. In this work, we explore a recently introduced entropy formula for a class of stochastic processes with more than one absorbing…
Conditional mutual information is important in the selection and interpretation of graphical models. Its empirical version is well known as a generalised likelihood ratio test and that it may be represented as a difference in entropy. We…
Redundancy of experimental data is the basic statistic from which the complexity of a natural phenomenon and the proper number of experiments needed for its exploration can be estimated. The redundancy is expressed by the entropy of…
We show that macroscopic irreversible thermodynamics for viscous fluids can be derived from exact information-theoretic thermodynamic identities valid at the microscale. Entropy production, in particular, is a measure of the loss of…
Measurements can be viewed as interactions between a measured system and a pointer system that imprint information about the system on the pointer. For so-called unbiased interactions, the measurement statistics--the information…
We study an opinion formation model by the means of a co-evolving complex network where the vertices represent the individuals, characterised by their evolving opinions, and the edges represent the interactions among them. The network…
Williams and Beer (2010) proposed a nonnegative mutual information decomposition, based on the construction of information gain lattices, which allows separating the information that a set of variables contains about another into components…
We examine a class of deep learning models with a tractable method to compute information-theoretic quantities. Our contributions are three-fold: (i) We show how entropies and mutual informations can be derived from heuristic statistical…
Entropy production is arguably the most universally applicable measure of non-equilibrium behavior, particularly for systems coupled to a heat bath. This setting encompasses driven soft matter as well as biomolecular, biochemical, and…