相关论文: Dynamic metastability in the self-attention model

We develop a rigorous treatment of discontinuous stochastic unitary evolution for a system of quantum particles that interacts singularly with quantum "bubbles" at random instants of time. This model of a "cloud chamber" allows to watch and…

This article deals with the issues of global-in-time existence and asymptotic analysis of a fluid-particle interaction model in the so-called bubbling regime. The mixture occupies the physical space $\Omega \subset \mathbb{R}^3$ which may…

Attention mechanisms are widely used in artificial intelligence to enhance performance and interpretability. In this paper, we investigate their utility in modeling classical dynamical systems -- specifically, a noisy predator-prey…

Latest development of neural models has connected the encoder and decoder through a self-attention mechanism. In particular, Transformer, which is solely based on self-attention, has led to breakthroughs in Natural Language Processing (NLP)…

The self-attention mechanism has been a key factor in the advancement of vision Transformers. However, its quadratic complexity imposes a heavy computational burden in high-resolution scenarios, restricting the practical application.…

A continuum model for self-organized dynamics is numerically investigated. The model describes systems of particles subject to alignment interaction and short-range repulsion. It consists of a non-conservative hyperbolic system for the…

We revisit motility-induced phase separation in two models of active particles interacting by pairwise repulsion. We show that the resulting dense phase contains gas bubbles distributed algebraically up to a typically large cutoff scale. At…

Numerical simulations in two space dimensions are used to examine the dynamics, transport, and equilibrium behaviors of a neutrally buoyant circular object immersed in an active suspension within a larger closed circular container. The…

We propose a method for learning dynamical systems from high-dimensional empirical data that combines variational autoencoders and (spatio-)temporal attention within a framework designed to enforce certain scientifically-motivated…

While attention has been empirically shown to improve model performance, it lacks a rigorous mathematical justification. This short paper establishes a novel connection between attention mechanisms and multinomial regression. Specifically,…

We study the performance of Markov chains for the $q$-state ferromagnetic Potts model on random regular graphs. It is conjectured that their performance is dictated by metastability phenomena, i.e., the presence of "phases" (clusters) in…

A granular system confined in a quasi two-dimensional box that is vertically vibrated can transit to an absorbing state in which all particles bounce vertically in phase with the box, with no horizontal motion. In principle, this state can…

Modern recommender systems employ various sequential modules such as self-attention to learn dynamic user interests. However, these methods are less effective in capturing collaborative and transitional signals within user interaction…

We discuss a resource-competition model, which takes the MacArthur's model as a platform, to unveil interesting connections with glassy features and jamming in high dimension. This model presents two qualitatively different phases: a…

Driven suspensions, where energy is input at a particle scale, are a framework for understanding general principles of out-of-equilibrium organization. A large number of simple interacting units can give rise to non-trivial structure and…

A simple three-dimensional model of a fluid whose constituent particles interact via a short range attractive and long range repulsive potential is used to model the aggregation into large spherical-like clusters made up of hundreds of…

Attention, specifically scaled dot-product attention, has proven effective for natural language, but it does not have a mechanism for handling hierarchical patterns of arbitrary nesting depth, which limits its ability to recognize certain…

We derive a variational principle for the dynamical stability of a cluster as a gas sphere in a box. Newtonian clusters are always dynamically stable and, for relativistic clusters, the relation between dynamical and thermodynamical…

We consider a long-range interacting system of $N$ particles moving on a spherical surface under an attractive Heisenberg-like interaction of infinite range, and evolving under deterministic Hamilton dynamics. The system may also be viewed…

Understanding the influence of activity on dense amorphous assemblies is crucial for biological processes such as wound healing, embryogenesis, or cancer progression. Here, we study the effect of self-propulsion forces of amplitude $f_0$…