Related papers: Information decomposition in complex systems via m…
The Information bottleneck method is an unsupervised non-parametric data organization technique. Given a joint distribution P(A,B), this method constructs a new variable T that extracts partitions, or clusters, over the values of A that are…
This thesis uses a quantity that is defined and justified by information theory -- mutual information -- to examine models of condensed matter systems. More precisely, it studies models which are made up out of ferromagnetically interacting…
We use a well known model (T. Vicsek et al. Phys Rev Lett 15, 1226 (1995)) for flocking to test mutual information as a tool for detecting order-disorder transitions, in particular when observations of the system are limited. We show that…
Consider a joint quantum state of a system and its environment. A measurement on the environment induces a decomposition of the system state. Using algorithmic information theory, we define the preparation information of a pure or mixed…
Dictionary learning is a challenge topic in many image processing areas. The basic goal is to learn a sparse representation from an overcomplete basis set. Due to combining the advantages of generic multiscale representations with learning…
Machine-learning models have recently encountered enormous success for predicting the properties of materials. These are often trained based on data that present various levels of accuracy, with typically much less high- than low-fidelity…
During the last few years an area of active research in the field of complex systems is that of their information storing and processing abilities. Common opinion has it that the most interesting beaviour of these systems is found ``at the…
The concept of information has emerged as a language in its own right, bridging several disciplines that analyze natural phenomena and man-made systems. Integrated information has been introduced as a metric to quantify the amount of…
For better learning, large datasets are often split into small batches and fed sequentially to the predictive model. In this paper, we study such batch decompositions from a probabilistic perspective. We assume that data points (possibly…
We offer a new approach to the information decomposition problem in information theory: given a 'target' random variable co-distributed with multiple 'source' variables, how can we decompose the mutual information into a sum of non-negative…
Exploiting the theory of state space models, we derive the exact expressions of the information transfer, as well as redundant and synergistic transfer, for coupled Gaussian processes observed at multiple temporal scales. All of the terms,…
In the 21st century, many of the crucial scientific and technical issues facing humanity can be understood as problems associated with understanding, modelling, and ultimately controlling complex systems: systems comprised of a large number…
Advances in information technology reduce barriers to information propagation, but at the same time they also induce the information overload problem. For the making of various decisions, mere digestion of the relevant information has…
This paper presents the foundation for a decomposition theory for Boolean networks, a type of discrete dynamical system that has found a wide range of applications in the life sciences, engineering, and physics. Given a Boolean network…
Feature selection is one of the most fundamental problems in machine learning. An extensive body of work on information-theoretic feature selection exists which is based on maximizing mutual information between subsets of features and class…
We derive three fundamental decompositions on relevant information quantities in feedback systems. The feedback systems considered in this paper are only restricted to be causal in time domain and the channels are allowed to be subject to…
This short paper presents the potential of using machine learning to predict materials behaviour in the context of hydrogen interaction with steel. Effort has been made to understand the quality, and amount of data needed to get improved…
Since its introduction, the partial information decomposition (PID) has emerged as a powerful, information-theoretic technique useful for studying the structure of (potentially higher-order) interactions in complex systems. Despite its…
Complex systems are found in most branches of science. It is still argued how to best quantify their complexity and to what end. One prominent measure of complexity (the statistical complexity) has an operational meaning in terms of the…
It is known that statistical model selection as well as identification of dynamical equations from available data are both very challenging tasks. Physical systems behave according to their underlying dynamical equations which, in turn, can…