Related papers: Can Transformers predict system collapse in dynami…
We show that a neural network originally designed for language processing can learn the dynamical rules of a stochastic system by observation of a single dynamical trajectory of the system, and can accurately predict its emergent behavior…
We demonstrate that transformers obtain impressive performance even when some of the layers are randomly initialized and never updated. Inspired by old and well-established ideas in machine learning, we explore a variety of non-linear…
Artificial Recurrent Neural Networks are a powerful information processing abstraction, and Reservoir Computing provides an efficient strategy to build robust implementations by projecting external inputs into high dimensional dynamical…
An artificial neural network architecture, parameterization networks, is proposed for simulating extrapolated dynamics beyond observed data in dynamical systems. Parameterization networks are used to ensure the long term integrity of…
A minimal model for reservoir computing is studied. We demonstrate that a reservoir computer exists that emulates given coupled maps by constructing a modularized network. We describe a possible mechanism for collapses of the emulation in…
Mixture models arise in many regression problems, but most methods have seen limited adoption partly due to these algorithms' highly-tailored and model-specific nature. On the other hand, transformers are flexible, neural sequence models…
Reservoir computing is a very promising approach for the prediction of complex nonlinear dynamical systems. Besides capturing the exact short-term trajectories of nonlinear systems, it has also proved to reproduce its characteristic…
Recent research has demonstrated Reservoir Computing's capability to model various chaotic dynamical systems, yet its application to Hamiltonian systems remains relatively unexplored. This paper investigates the effectiveness of Reservoir…
Biological systems are promising substrates for computation because they naturally process environmental information through complex internal dynamics. In this study, we investigate whether bacterial metabolic models can act as physical…
Prediction models that capture and use the structure of state-space dynamics can be very effective. In practice, however, one rarely has access to full information about that structure, and accurate reconstruction of the dynamics from…
The identification of nonlinear dynamics from observations is essential for the alignment of the theoretical ideas and experimental data. The last, in turn, is often corrupted by the side effects and noise of different natures, so…
Despite their empirical success, pushing Transformer architectures to extreme depth often leads to a paradoxical failure: representations become increasingly redundant, lose rank, and ultimately collapse. Existing explanations largely…
In this paper, we discuss a novel model reduction framework for generalized linear systems. The transfer functions of these systems are assumed to have a special structure, e.g., coming from second-order linear systems and time-delay…
Despite the importance of quantifying how the spatial patterns of extreme precipitation will change with warming, we lack tools to objectively analyze the storm-scale outputs of modern climate models. To address this gap, we develop an…
Forecasting system behaviour near and across bifurcations is crucial for identifying potential shifts in dynamical systems. While machine learning has recently been used to learn critical transitions and bifurcation structures from data,…
Transformer architectures have revolutionized machine learning across a wide range of domains, from natural language processing to scientific computing. However, their growing deployment in high-stakes applications, such as computer vision,…
The basic strategy underlying models of spontaneous wave function collapse (collapse models) is to modify the Schroedinger equation by including nonlinear stochastic terms, which tend to localize wave functions in space in a dynamical…
Selecting or designing an appropriate domain adaptation algorithm for a given problem remains challenging. This paper presents a Transformer model that can provably approximate and opt for domain adaptation methods for a given dataset in…
Reservoir Computing offers a great computational framework where a physical system can directly be used as computational substrate. Typically a "reservoir" is comprised of a large number of dynamical systems, and is consequently…
This paper is a preliminary work to address the problem of dynamical systems with parameters varying in time. An idea to predict their behaviour is proposed. These systems are called \emph{transient systems}, and are distinguished from…