Related papers: Localmax dynamics for attention in transformers an…
Topological mechanical metamaterials have enabled new ways to control stress and deformation propagation. Exemplified by Maxwell lattices, they have been studied extensively using a linearized formalism. Herein, we study a two-dimensional…
We consider the problem of output feedback stabilization in linear systems when the measured outputs and control inputs are subject to event-triggered sampling and dynamic quantization. A new sampling algorithm is proposed for outputs which…
Hyperexponential stability is investigated for dynamical systems with the use of both, explicit and implicit, Lyapunov function methods. A nonlinear hyperexponential control is designed for stabilizing linear systems. The tuning procedure…
This paper studies consensus of discrete-time multi-agent systems under time-varying directed communication, state and input constraints using a distributed multi-step model predictive control (MPC) framework. Consensus is recast as…
We consider stochastic systems of interacting particles or agents, with dynamics determined by an interaction kernel which only depends on pairwise distances. We study the problem of inferring this interaction kernel from observations of…
In recent years, attention mechanisms have significantly enhanced the performance of object detection by focusing on key feature information. However, prevalent methods still encounter difficulties in effectively balancing local and global…
Attention mechanisms have revolutionized several domains of artificial intelligence, such as natural language processing and computer vision, by enabling models to selectively focus on relevant parts of the input data. While recent work has…
Since its inception in "Attention Is All You Need", transformer architecture has led to revolutionary advancements in NLP. The attention layer within the transformer admits a sequence of input tokens $X$ and makes them interact through…
Human joint dynamic stiffness plays an important role in the stability of performance augmentation exoskeletons. In this paper, we consider a new frequency domain model of the human joint dynamics which features a complex value stiffness.…
In this paper, we propose a dynamical systems perspective of the Expectation-Maximization (EM) algorithm. More precisely, we can analyze the EM algorithm as a nonlinear state-space dynamical system. The EM algorithm is widely adopted for…
Estimating optimal dynamic policies from offline data is a fundamental problem in dynamic decision making. In the context of causal inference, the problem is known as estimating the optimal dynamic treatment regime. Even though there exists…
Variable count is among the main scalability bottlenecks for transformer modeling in multivariate time series (MTS) data. On top of this, a growing consensus in the field points to indiscriminate inter-variable mixing as a potential source…
We consider the data-driven stabilization of discrete-time linear time-varying systems. The controller is defined as a linear state-feedback law whose gain is adapted to the plant changes through a data-based event-triggering rule. To do…
The softmax function is crucial in Transformer attention, which normalizes each row of the attention scores with summation to one, achieving superior performances over other alternative functions. However, the softmax function can face a…
In this paper, we study the stability problem of a stochastic, nonlinear, discrete-time system. We introduce a linear transfer operator-based Lyapunov measure as a new tool for stability verification of stochastic systems. Weaker…
As saturated output observations are ubiquitous in practice, identifying stochastic systems with such nonlinear observations is a fundamental problem across various fields. This paper investigates the asymptotically efficient identification…
There have been many recent efforts to study accelerated optimization algorithms from the perspective of dynamical systems. In this paper, we focus on the robustness properties of the time-varying continuous-time version of these dynamics.…
We present an extension of local sensitivity analysis, also referred to as the perturbation approach for uncertainty quantification, to Bayesian inverse problems. More precisely, we show how moments of random variables with respect to the…
Recently, the robustness of deep learning models has received widespread attention, and various methods for improving model robustness have been proposed, including adversarial training, model architecture modification, design of loss…
Markov Decision Processes (MDPs) are a formal framework for modeling and solving sequential decision-making problems. In finite-time horizons such problems are relevant for instance for optimal stopping or specific supply chain problems,…