Related papers: An Information-Theoretic Formalism for Multiscale …
When dealing with evolving or multi-dimensional complex systems, network theory provides with elegant ways of describing their constituting components, through respectively time-varying and multi-layer complex networks. Nevertheless, the…
The correlations that can be observed between a set of variables depend on the causal structure underpinning them. Causal structures can be modeled using directed acyclic graphs, where nodes represent variables and edges denote functional…
We introduce an asymmetric distance in the space of learning tasks, and a framework to compute their complexity. These concepts are foundational for the practice of transfer learning, whereby a parametric model is pre-trained for a task,…
Extracting higher-order structures from multivariate data has become an area of intensive study in complex systems science, as these multipartite interactions can reveal insights into fundamental features of complex systems like emergent…
There is a lack of formalism for some key foundational concepts in systems engineering. One of the most recently acknowledged deficits is the inadequacy of systems engineering practices for engineering intelligent systems. In our previous…
Integrated Information Theory is one of the leading models of consciousness. It aims to describe both the quality and quantity of the conscious experience of a physical system, such as the brain, in a particular state. In this contribution,…
Understanding the dynamical behavior of complex systems is of exceptional relevance in everyday life, from biology to economy. In order to describe the dynamical organization of complex systems, existing methods require the knowledge of the…
Integrated information theory (IIT) starts from consciousness itself and identifies a set of properties (axioms) that are true of every conceivable experience. The axioms are translated into a set of postulates about the substrate of…
We define the class of multivariate group entropies as a novel set of information - theoretical measures, which extends significantly the family of group entropies. We propose new examples related to the "super-exponential" universality…
This study investigates entropy's potential for analyzing scientific research patterns across disciplines. Originating from thermodynamics, entropy now measures uncertainty and diversity in information systems. We examine Shannon Entropy,…
Information Theory concepts and methodologies conform the background of how communication systems are studied and understood. They are mainly focused on the source-channel-receiver problem and on the asymptotic limits of accuracy and…
Network theory provides tools which are particularly appropriate for assessing the complex interdependencies that characterise our modern connected world. This article presents an introduction to network theory, in a way that doesn't…
Distributed systems, such as biological and artificial neural networks, process information via complex interactions engaging multiple subsystems, resulting in high-order patterns with distinct properties across scales. Investigating how…
We propose a unified theoretical framework for quantifying spatio-temporal interactions in a stochastic dynamical system based on information geometry. In the proposed framework, the degree of interactions is quantified by the divergence…
The standard assumptions that underlie many conceptual and quantitative frameworks do not hold for many complex physical, biological, and social systems. Complex systems science clarifies when and why such assumptions fail and provides…
Kolmogorov argued that the concept of information exists also in problems with no underlying stochastic model (as Shannon's information representation) for instance, the information contained in an algorithm or in the genome. He introduced…
Complex adaptive systems (CAS) can be described as systems of information flows dynamically interacting across scales in order to adapt and survive. CAS often consist of many components that work towards a shared goal, and interact across…
To understand the structure of a large-scale biological, social, or technological network, it can be helpful to decompose the network into smaller subunits or modules. In this article, we develop an information-theoretic foundation for the…
The quantification of the complexity of networks is, today, a fundamental problem in the physics of complex systems. A possible roadmap to solve the problem is via extending key concepts of information theory to networks. In this paper we…
Despite the remarkable success of large large-scale neural networks, we still lack unified notation for thinking about and describing their representational spaces. We lack methods to reliably describe how their representations are…