Related papers: Data-driven model order reduction of linear switch…

We introduce a constrained optimization framework for training transformers that behave like optimization descent algorithms. Specifically, we enforce layerwise descent constraints on the objective function and replace standard empirical…

Time series prediction plays a crucial role in various industrial fields. In recent years, neural networks with a transformer backbone have achieved remarkable success in many domains, including computer vision and NLP. In time series…

Recurrent Neural Networks were, until recently, one of the best ways to capture the timely dependencies in sequences. However, with the introduction of the Transformer, it has been proven that an architecture with only attention-mechanisms…

Dynamical systems with high intrinsic dimensionality are often characterized by extreme events having the form of rare transitions several standard deviations away from the mean. For such systems, order-reduction methods through projection…

Koopman analysis provides a general framework from which to analyze a nonlinear dynamical system in terms of a linear operator acting on an infinite-dimensional observable space. This theoretical framework provides a rigorous underpinning…

In the article, we discuss the architecture of the polynomial neural network that corresponds to the matrix representation of Lie transform. The matrix form of Lie transform is an approximation of the general solution of the nonlinear…

A method for model reduction in nonlinear ODE systems is demonstrated through computational examples. The method does not require an implicit separation of time-scales in the fine dynamics to be effective. From the computational standpoint,…

Generative models such as denoising diffusion models are quickly advancing their ability to approximate highly complex data distributions. They are also increasingly leveraged in scientific machine learning, where samples from the implied…

In this paper, an end-to-end nonlinear model reduction methodology is presented based on the convolutional recurrent autoencoder networks. The methodology is developed in the context of the overall data-driven reduced-order model framework…

Classifiers that are linear in their parameters, and trained by optimizing a convex loss function, have predictable behavior with respect to changes in the training data, initial conditions, and optimization. Such desirable properties are…

Linear Transformers and State Space Models have emerged as efficient alternatives to softmax Transformers for causal sequence modeling, enabling parallel training via matrix multiplication and efficient RNN-style inference. However, despite…

This paper proposes a data-driven framework to solve time-varying optimization problems associated with unknown linear dynamical systems. Making online control decisions to regulate a dynamical system to the solution of an optimization…

A general class of unidirectional transforms is presented that can be computed in a distributed manner along an arbitrary routing tree. Additionally, we provide a set of conditions under which these transforms are invertible. These…

In this paper, we propose a constrained linear data-feature mapping model as an interpretable mathematical model for image classification using convolutional neural network (CNN) such as the ResNet. From this viewpoint, we establish the…

Linear regression on network-linked observations has been an essential tool in modeling the relationship between response and covariates with additional network structures. Previous methods either lack inference tools or rely on restrictive…

State-space models effectively model multivariate time series by updating over time a representation of the system state from which predictions are made. The state representation is usually a vector without any explicit structure.…

In this work, the problem of designing observers for estimating a single nonlinear functional of the state is formulated for general nonlinear systems. Notions of functional observer linearization are also formulated, in terms achieving…

Networks are powerful tools for modeling interactions in complex systems. While traditional networks use scalar edge weights, many real-world systems involve multidimensional interactions. For example, in social networks, individuals often…

Boolean networks are extensively applied as models of complex dynamical systems, aiming at capturing essential features related to causality and synchronicity of the state changes of components along time. Dynamics of Boolean networks…

Nonlinear systems play a significant role in numerous scientific and engineering disciplines, and comprehending their behavior is crucial for the development of effective control and prediction strategies. This paper introduces a novel…