Related papers: Hidden information and regularities of information…
The Conservation of Energy plays a pivotal part in the development of the physical sciences. With the growth of computation and the study of other discrete token based systems such as the genome, it is useful to ask if there are…
Markov Random Field models are powerful tools for the study of complex systems. However, little is known about how the interactions between the elements of such systems are encoded, especially from an information-theoretic perspective. In…
We propose an information-theoretic framework for analyzing control systems based on the close relationship of controllers to communication channels. A communication channel takes an input state and transforms it into an output state. A…
Despite the wide usage of information as a concept in science, we have yet to develop a clear & concise scientific definition. This paper is aimed at laying the foundations for a new theory concerning the mechanics of information alongside…
Conditional mutual information is important in the selection and interpretation of graphical models. Its empirical version is well known as a generalised likelihood ratio test and that it may be represented as a difference in entropy. We…
Imitation is a basic updating mechanism for strategy evolution in structured populations, determining how individuals sample social information and translate it into behavioral changes. Higher-order networks, such as hypergraphs, generalize…
We incorporate into the empirical measure the auxiliary information given by a finite collection of expectation in an optimal information geometry way. This allows to unify several methods exploiting a side information and to uniquely…
In a genetic algorithm, fluctuations of the entropy of a genome over time are interpreted as fluctuations of the information that the genome's organism is storing about its environment, being this reflected in more complex organisms. The…
We propose an application of molecular information theory to analyze the folding of single domain proteins. We analyze results from various areas of protein science, such as sequence-based potentials, reduced amino acid alphabets, backbone…
We simulate the Vicsek model utilising topological neighbour interactions and estimate information theoretic quantities as a function of noise, the variability in the extent to which each animal aligns with its neighbours, and the flock…
Learning compact discrete representations of data is a key task on its own or for facilitating subsequent processing of data. In this paper we present a model that produces Discrete InfoMax Codes (DIMCO); we learn a probabilistic encoder…
The performance of imitation learning policies often hinges on the datasets with which they are trained. Consequently, investment in data collection for robotics has grown across both industrial and academic labs. However, despite the…
The information implicitly represented in the state of physical systems allows one to analyze them with analytical techniques from statistical mechanics and information theory. In the case of complex networks such techniques are inspired by…
This paper introduces a bilateral matching mechanism to explain why different populations have different levels of cooperation. The traditional game theory assumes that individuals can acquire their neighbor's information without cost after…
The principle of maximum entropy is a broadly applicable technique for computing a distribution with the least amount of information possible while constrained to match empirically estimated feature expectations. However, in many real-world…
Recent progress in experimental techniques has enabled us to quantitatively study stochastic and flexible behavior of biological systems. For example, gene regulatory networks perform stochastic information processing and their…
We develop numerical and analytical approaches to calculate mutual information between complete paths of two molecular components embedded into a larger reaction network. In particular, we focus on a continuous-time Markov chain formalism,…
Can we learn more from data than existed in the generating process itself? Can new and useful information be constructed from merely applying deterministic transformations to existing data? Can the learnable content in data be evaluated…
We look at chaotic systems evolving in fractal phase space. The entropy change in time due to the fractal geometry is assimilated to the information growth through the scale refinement. Due to the incompleteness, at any scale, of the…
Assessing the synergistic high-order behaviors (HOBs) that emerge from underlying structural mechanisms is crucial to characterize complex systems. This work leverages the combined use of predictability and information measures to detect…