Related papers: Innovation Capacity of Dynamical Learning Systems

We consider a system of $M$ particles in contact with a heat reservoir of $N\gg M$ particles. The evolution in the system and the reservoir, together with their interaction, are modeled via the Kac's Master Equation. We chose the initial…

The Free Energy Principle (FEP) is a leading framework for mathematically modeling self-organization and learning, while Integrated Information Theory (IIT) is a computational ontology of consciousness oriented around irreducible cause and…

Wireless information-centric networks consider storage as one of the network primitives, and propose to cache data within the network in order to improve latency and reduce bandwidth consumption. We study the throughput capacity and delay…

Discovering conservation laws for a given dynamical system is important but challenging. In a theorist setup (differential equations and basis functions are both known), we propose the Sparse Invariant Detector (SID), an algorithm that…

We develop dynamical non-Markovian description of quantum computing in weak coupling limit, in lowest order approximation. We show that long range memory of quantum reservoir produces strong interrelation between structure of noise and…

Wireless information-centric networks consider storage as one of the network primitives, and propose to cache data within the network in order to improve latency and reduce bandwidth consumption. We study the throughput capacity and latency…

Coping with the intermittency of renewables is a fundamental challenge, with load shifting and grid-scale storage as key responses. We propose Information Batteries (IB), in which energy is stored in the form of information -- specifically,…

Numerical models of weather and climate critically depend on long-term stability of integrators for systems of hyperbolic conservation laws. While such stability is often obtained from (physical or numerical) dissipation terms, physical…

This manuscript investigates the information-theoretic limits of integrated sensing and communications (ISAC), aiming for simultaneous reliable communication and precise channel state estimation. We model such a system with a…

The problem of distributed representation learning is one in which multiple sources of information $X_1,\ldots,X_K$ are processed separately so as to learn as much information as possible about some ground truth $Y$. We investigate this…

Information Pursuit (IP) is an explainable prediction algorithm that greedily selects a sequence of interpretable queries about the data in order of information gain, updating its posterior at each step based on observed query-answer pairs.…

Preserving information stored in a physical system subjected to noise can be modeled in a communication-theoretic paradigm, in which storage and retrieval correspond to an input encoding and output decoding, respectively. The encoding and…

Filter pruning has gained widespread adoption for the purpose of compressing and speeding up convolutional neural networks (CNNs). However, existing approaches are still far from practical applications due to biased filter selection and…

Reservoir computing is a popular approach to design recurrent neural networks, due to its training simplicity and approximation performance. The recurrent part of these networks is not trained (e.g., via gradient descent), making them…

Reservoir computing (RC) is a state-of-the-art machine learning method that makes use of the power of dynamical systems (the reservoir) for real-time inference. When using biological complex systems as reservoir substrates, it serves as a…

Reservoir computing is a form of machine learning that utilizes nonlinear dynamical systems to perform complex tasks in a cost-effective manner when compared to typical neural networks. Many recent advancements in reservoir computing, in…

Invariants and conservation laws convey critical information about the underlying dynamics of a system, yet it is generally infeasible to find them from large-scale data without any prior knowledge or human insight. We propose ConservNet to…

Current deep learning models often suffer from catastrophic forgetting of old knowledge when continually learning new knowledge. Existing strategies to alleviate this issue often fix the trade-off between keeping old knowledge (stability)…

Associative memory, traditionally modeled by Hopfield networks, enables the retrieval of previously stored patterns from partial or noisy cues. Yet, the local computational principles which are required to enable this function remain…

Physical reservoir computing has recently been attracting attention for its ability to significantly reduce the computational resources required to process time-series data. However, the physical reservoirs that have been reported to date…