Related papers: Recurrent Independent Mechanisms
Recent work has shown that recurrent neural networks (RNNs) can implicitly capture and exploit hierarchical information when trained to solve common natural language processing tasks such as language modeling (Linzen et al., 2016) and…
A wireless communication system is studied that operates in the presence of multiple reconfigurable intelligent surfaces (RISs). In particular, a multi-operator environment is considered where each operator utilizes an RIS to enhance its…
We introduce a general and simple structural design called Multiplicative Integration (MI) to improve recurrent neural networks (RNNs). MI changes the way in which information from difference sources flows and is integrated in the…
Real-world systems are characterized by complex interactions of their internal degrees of freedom, while living in ever-changing environments whose net effect is to act as additional couplings. Here, we introduce a paradigmatic interacting…
Many important phenomena in biochemistry and biology exploit dynamical features such as multi-stability, oscillations, and chaos. Construction of novel chemical systems with such rich dynamics is a challenging problem central to the fields…
An important development in deep learning from the earliest MLPs has been a move towards architectures with structural inductive biases which enable the model to keep distinct sources of information and routes of processing well-separated.…
Recurrent neural networks (RNNs) for reinforcement learning (RL) have shown distinct advantages, e.g., solving memory-dependent tasks and meta-learning. However, little effort has been spent on improving RNN architectures and on…
We introduce Invariant Risk Minimization (IRM), a learning paradigm to estimate invariant correlations across multiple training distributions. To achieve this goal, IRM learns a data representation such that the optimal classifier, on top…
In this paper, we present an approach for Recurrent Iterative Gating called RIGNet. The core elements of RIGNet involve recurrent connections that control the flow of information in neural networks in a top-down manner, and different…
Biochemistry, ecology, and neuroscience are examples of prominent fields aiming at describing interacting systems that exhibit non-trivial couplings to complex, ever-changing environments. We have recently shown that linear interactions and…
Reactive synthesis is a paradigm for automatically building correct-by-construction systems that interact with an unknown or adversarial environment. We study how to do reactive synthesis when part of the specification of the system is that…
Recurrent neural networks (RNNs) have been used extensively and with increasing success to model various types of sequential data. Much of this progress has been achieved through devising recurrent units and architectures with the…
Statistical learning relies upon data sampled from a distribution, and we usually do not care what actually generated it in the first place. From the point of view of causal modeling, the structure of each distribution is induced by…
A core aspect of human intelligence is the ability to learn new tasks quickly and switch between them flexibly. Here, we describe a modular continual reinforcement learning paradigm inspired by these abilities. We first introduce a visual…
One of the defining features of living systems is their adaptability to changing environmental conditions. This requires organisms to extract temporal and spatial features of their environment, and use that information to compute the…
Reconfigurable intelligent surfaces (RISs) are an emerging technology for enhancing the performance of wireless networks at a low and affordable cost, complexity, and power consumption. We introduce an algorithm for optimizing a…
Exact inference in complex probabilistic models often incurs prohibitive computational costs. This challenge is particularly acute for autonomous agents in dynamic environments that require frequent, real-time belief updates. Existing…
The scarcity of high-quality training data presents a fundamental bottleneck to scaling machine learning models. This challenge is particularly acute in recommendation systems, where extreme sparsity in user interactions leads to rugged…
Recursive InterNetwork Architecture is a clean-slate approach to how to deal with the current issues of the Internet based on the traditional TCP/IP networking stack. Instead of using a fixed number of layers with dedicated functionality,…
We propose a recurrent extension of the Ladder networks whose structure is motivated by the inference required in hierarchical latent variable models. We demonstrate that the recurrent Ladder is able to handle a wide variety of complex…