Related papers: Attention Mechanisms in Dynamical Systems: A Case …

Attention is fundamental to cognition, yet it remains a challenge to understand attention in tasks approaching real-world complexity. Here, we approached this problem by modeling gaze patterns of monkeys playing Pac-Man. We first show a…

Attention mechanism has become the dominant module in natural language processing models. It is computationally intensive and depends on massive power-hungry multiplications. In this paper, we rethink variants of attention mechanism from…

Although attention mechanisms have been applied to a variety of deep learning models and have been shown to improve the prediction performance, it has been reported to be vulnerable to perturbations to the mechanism. To overcome the…

Attention mechanisms represent a fundamental paradigm shift in neural network architectures, enabling models to selectively focus on relevant portions of input sequences through learned weighting functions. This monograph provides a…

To improve the robustness of transformer neural networks used for temporal-dynamics prediction of chaotic systems, we propose a novel attention mechanism called easy attention which we demonstrate in time-series reconstruction and…

Attention mechanisms are ubiquitous components in neural architectures applied to natural language processing. In addition to yielding gains in predictive accuracy, attention weights are often claimed to confer interpretability, purportedly…

Urban ecosystems exhibit complex predator-prey dynamics increasingly disrupted by anthropogenic disturbances (e.g., noise, habitat fragmentation). Classical Lotka-Volterra (LV) models fail to capture these human-induced stressors, and…

Attention is an increasingly popular mechanism used in a wide range of neural architectures. The mechanism itself has been realized in a variety of formats. However, because of the fast-paced advances in this domain, a systematic overview…

We consider a broad class of stochastic lattice predator-prey models, whose main features are overviewed. In particular, this article aims at drawing a picture of the influence of spatial fluctuations, which are not accounted for by the…

The Transformer is a sequence model that forgoes traditional recurrent architectures in favor of a fully attention-based approach. Besides improving performance, an advantage of using attention is that it can also help to interpret a model…

The broad application range of the predator-prey modelling enabled us to apply it to represent the dynamics of the work-employment system. For the adopted period, we conclude that this dynamics is chaotic in the beginning of the time series…

The forces which drive growth, development, survival and change within an ecological system involving a predator and prey specie are not easily addressed in the field. To better understand the dynamics in the system, ecologists have turned…

We propose a novel attention model that can accurately attends to target objects of various scales and shapes in images. The model is trained to gradually suppress irrelevant regions in an input image via a progressive attentive process…

Time series prediction with deep learning methods, especially long short-term memory neural networks (LSTMs), have scored significant achievements in recent years. Despite the fact that the LSTMs can help to capture long-term dependencies,…

We study the stochastic spatial Lotka-Volterra (LV) model for predator-prey interaction subject to a periodically varying carrying capacity. The LV model with on-site lattice occupation restrictions that represent finite food resources for…

Attention is an important cognition process of humans, which helps humans concentrate on critical information during their perception and learning. However, although many machine learning models can remember information of data, they have…

The attention mechanism is considered the backbone of the widely-used Transformer architecture. It contextualizes the input by computing input-specific attention matrices. We find that this mechanism, while powerful and elegant, is not as…

The Lotka-Volterra predator-prey model still represents the paradigm for the description of the competition in population dynamics. Despite its extreme simplicity, it does not admit an analytical solution, and for this reason, numerical…

Neural attention has become central to many state-of-the-art models in natural language processing and related domains. Attention networks are an easy-to-train and effective method for softly simulating alignment; however, the approach does…

This paper presents a study of the two-predators-two-preys discrete-time Lotka-Volterra model with self- inhibition terms for preys with direct applications to ecological problems. Parameters in the model are modified so that each of them…