Related papers: Remarks on Feedforward Circuits, Adaptation, and P…
Organisms gain by anticipating future changes in the environment. Those environmental changes often follow stochastic trends. The greater the slope of the trend, the more likely the trend's momentum carries the future trend in the same…
Stability theory plays a crucial role in feedback control. However, adaptive control theory requires advanced and specialized stability notions that are not frequently used in standard feedback control theory. The present document is a set…
The nonlinear propagation of ultrashort pulses in optical fiber depends sensitively on both input pulse and fiber parameters. As a result, optimizing propagation for specific applications generally requires time-consuming simulations based…
We present a theoretical formalism to study steady state information transmission in type 1 coherent feed-forward loop motif with an additive signal integration mechanism. Our construct allows a two-step cascade to be slowly transformed…
The well-known generalization problem hinders the application of artificial neural networks in continuous-time prediction tasks with varying latent dynamics. In sharp contrast, biological systems can neatly adapt to evolving environments…
Signal processing in biological systems is delicately executed by specialised networks, which are modular assemblies of network motifs. The motifs are independently functional circuits found in enormous numbers in any living cell. A very…
Unknown nonlinear dynamics can limit the performance of model-based feedforward control. The aim of this paper is to develop a feedforward control framework for systems with unknown, typically nonlinear, dynamics. To address the unknown…
We introduce a new structure for memory neural networks, called feedforward sequential memory networks (FSMN), which can learn long-term dependency without using recurrent feedback. The proposed FSMN is a standard feedforward neural…
This paper explores the properties of adaptive systems with closed-loop reference models. Using additional design freedom available in closed-loop reference models, we design new adaptive controllers that are (a) stable, and (b) have…
This work presents an analysis of the effectiveness of using standard shallow feed-forward networks to mimic the behavior of the attention mechanism in the original Transformer model, a state-of-the-art architecture for sequence-to-sequence…
Feedback alignment and related weight-transport-free algorithms are often proposed as biologically plausible alternatives to backpropagation, yet they are typically formulated in discrete phases with implicitly synchronized forward and…
Understanding cellular response to mechanical forces is immensely important for a plethora of biological processes. Focal adhesions are multi-molecular protein assemblies that connect the cell to the extracellular matrix and play a pivotal…
Transformers are ubiquitous in wide tasks. Interpreting their internals is a pivotal goal. Nevertheless, their particular components, feed-forward (FF) blocks, have typically been less analyzed despite their substantial parameter amounts.…
Many living and artificial systems improve their fitness or performance by adapting to changing environments or diverse training data. However, it remains unclear how such environmental variation influences adaptation, what is learned in…
The remarkable capability of Transformers to do reasoning and few-shot learning, without any fine-tuning, is widely conjectured to stem from their ability to implicitly simulate a multi-step algorithms -- such as gradient descent -- with…
Sparse random networks contain structures that can be considered as diluted feed-forward networks. Modeling of cortical circuits has shown that feed-forward structures, if strongly pronounced compared to the embedding random network, enable…
Recurrent neural networks can learn complex transduction problems that require maintaining and actively exploiting a memory of their inputs. Such models traditionally consider memory and input-output functionalities indissolubly entangled.…
Time Series Foundation Models (TSFMs) have shown promising zero-shot generalization across diverse forecasting tasks. However, their robustness to continual adaptation remains underexplored. In this work, we investigate the extent to which…
A grid-feeding converter system is added to a novel power system transient simulation scheme based on frequency response optimized integrators considering second order derivative. The converter system and its implementation in the…
We consider two different systems exhibiting a continuous phase transition into an absorbing state. Both models belong to the same universality class, i.e., they are characterized by the same scaling functions and the same critical…