Related papers: Architecture of Information

We show that a rate of conditional Shannon entropy reduction, characterizing the learning of an internal process about an external process, is bounded by the thermodynamic entropy production. This approach allows for the definition of an…

Autonomous construction of deep neural network (DNNs) is desired for data streams because it potentially offers two advantages: proper model's capacity and quick reaction to drift and shift. While the self-organizing mechanism of DNNs…

Optical wireless communication offers unprecedented communication speeds that can support the massive use of the Internet on a daily basis. In indoor environments, optical wireless networks are usually multi-user multiple-input…

We numerically show that a deep neural network (DNN) can learn macroscopic thermodynamic laws purely from microscopic data. Using molecular dynamics simulations, we generate the data of snapshot images of gas particles undergoing adiabatic…

The adaptive and surprising emergent properties of biological materials self-assembled in far-from-equilibrium environments serve as an inspiration for efforts to design nanomaterials and their properties. In particular, controlling the…

Information engines can use structured environments as a resource to generate work by randomizing ordered inputs and leveraging the increased Shannon entropy to transfer energy from a thermal reservoir to a work reservoir. We give a broadly…

Organisms in nature have evolved to exhibit flexibility in face of changes to the environment and/or to themselves. Artificial neural networks (ANNs) have proven useful for controlling of artificial agents acting in environments. However,…

Artificial neural network (ANN) potentials enable the efficient large-scale atomistic modeling of complex materials with near first-principles accuracy. For molecular dynamics simulations, accurate energies and interatomic forces are a…

The Artificial Intelligence field is flooded with optimisation methods. In this paper, we change the focus to developing modelling methods with the aim of getting us closer to Artificial General Intelligence. To do so, we propose a novel…

A critical mystery in neuroscience lies in determining how anatomical structure impacts the complex functional dynamics of human thought. How does large-scale brain circuitry constrain states of neuronal activity and transitions between…

Active inference is a normative framework for explaining behaviour under the free energy principle -- a theory of self-organisation originating in neuroscience. It specifies neuronal dynamics for state-estimation in terms of a descent on…

Units of complex systems -- such as neurons in the brain or individuals in societies -- must communicate efficiently to function properly: e.g., allowing electrochemical signals to travel quickly among functionally connected neuronal areas…

Machine Learning methods and, in particular, Artificial Neural Networks (ANNs) have demonstrated promising capabilities in material constitutive modeling. One of the main drawbacks of such approaches is the lack of a rigorous frame based on…

Information theory is a mathematical theory of learning with deep connections with topics as diverse as artificial intelligence, statistical physics, and biological evolution. Many primers on information theory paint a broad picture with…

Topological neural networks (TNNs) are information processing architectures that model representations from data lying over topological spaces (e.g., simplicial or cell complexes) and allow for decentralized implementation through localized…

The thermodynamic definition of entropy can be extended to nonequilibrium systems based on its relation to information. To apply this definition in practice requires access to the physical system's microstates, which may be prohibitively…

The interactions between three or more random variables are often nontrivial, poorly understood, and yet, are paramount for future advances in fields such as network information theory, neuroscience, genetics and many others. In this work,…

Neural Networks (NNs) are vulnerable to adversarial examples. Such inputs differ only slightly from their benign counterparts yet provoke misclassifications of the attacked NNs. The required perturbations to craft the examples are often…

In modern deep neural networks, the learning dynamics of the individual neurons is often obscure, as the networks are trained via global optimization. Conversely, biological systems build on self-organized, local learning, achieving…

Artificial neural networks (ANNs) are powerful tools capable of approximating any arbitrary mathematical function, but their interpretability remains limited, rendering them as black box models. To address this issue, numerous methods have…