Related papers: Feedback-Gated Rectified Linear Units
Machine learning has recently developed novel approaches, mimicking the synapses of the human brain to achieve similarly efficient learning strategies. Such an approach retains the universality of standard methods, while attempting to…
Convolutional Neural Networks (CNNs) are a class of artificial neural networks whose computational blocks use convolution, together with other linear and non-linear operations, to perform classification or regression. This paper explores…
In the context of a time-varying multiuser multiple-input-multiple-output (MIMO) system, we design recursive least squares based adaptive predictors and differential quantizers to minimize the sum mean squared error of the overall system.…
In this paper we propose a methodology for stabilizing single-input single-output feedback linearizable systems when no system model is known and no prior data is available to identify a model. Conceptually, we have been greatly inspired by…
Synaptic plasticity is metabolically expensive, yet animals continuously update their internal models without exhausting energy reserves. However, when artificial neural networks are trained, the network parameters are typically updated on…
Feedback is essential for learning, but its effectiveness relies heavily on how well it engages students in the educational process. Generative AI offers novel opportunities to efficiently produce rich, formative feedback, ranging from…
In this work we propose lifted regression/reconstruction networks (LRRNs), which combine lifted neural networks with a guaranteed Lipschitz continuity property for the output layer. Lifted neural networks explicitly optimize an energy model…
In recent years, imitation learning using neural networks has enabled robots to perform flexible tasks. However, since neural networks operate in a feedforward structure, they do not possess a mechanism to compensate for output errors. To…
Quantum control plays an irreplaceable role in practical use of quantum computers. However, some challenges have to be overcome to find more suitable and diverse control parameters. We propose a promising and generalizable…
In this paper we demonstrate how certain structured feedback gains necessarily emerge as the optimal controller gains in two linear optimal control formulations for multi-agent systems. We consider the cases of linear optimal…
Attention-based neural networks such as transformers have revolutionized various fields such as natural language processing, genomics, and vision. Here, we demonstrate the use of transformers for quantum feedback control through both a…
Most communication systems use some form of feedback, often related to channel state information. The common models used in analyses either assume perfect channel state information at the receiver and/or noiseless state feedback links.…
Binary representation is desirable for its memory efficiency, computation speed and robustness. In this paper, we propose adjustable bounded rectifiers to learn binary representations for deep neural networks. While hard constraining…
Recent architectural developments have enabled recurrent neural networks (RNNs) to reach and even surpass the performance of Transformers on certain sequence modeling tasks. These modern RNNs feature a prominent design pattern: linear…
This work is a study of acoustic non-reciprocity exhibited by a passive one-dimensional linear waveguide incorporating two local strongly nonlinear, asymmetric gates. Two local nonlinear gates break the symmetry and linearity of the…
Recent research has shown that supervised learning can be an effective tool for designing optimal feedback controllers for high-dimensional nonlinear dynamic systems. But the behavior of these neural network (NN) controllers is still not…
Language Models (LMs) have shown impressive performance in various natural language tasks. However, when it comes to natural language reasoning, LMs still face challenges such as hallucination, generating incorrect intermediate reasoning…
The present paper studies a feedback regulation problem, which may be interpreted as an adaptive control problem, but has not yet been studied in the control literature. The problem, which arises in at least two different biological…
Unlike traditional artificial neural networks (ANNs), biological neuronal networks solve complex cognitive tasks with sparse neuronal activity, recurrent connections, and local learning rules. These mechanisms serve as design principles in…
Predictive coding is a promising theoretical framework in neuroscience for understanding information transmission and perception. It posits that the brain perceives the external world through internal models and updates these models under…