相关论文: Stochastic Parroting in Temporal Attention -- Regu…

Using functional magnetic resonance imaging (fMRI) and deep learning to explore functional brain networks (FBNs) has attracted many researchers. However, most of these studies are still based on the temporal correlation between the sources…

This paper studies the role of sinks and diagonal patterns as attention switch and anti-oversmoothing mechanisms. We analyze geometric conditions under which sinks can be represented, showing a necessary alignment between the embedding of…

We study theoretically a bistable systems with time-delayed feedback driven by weak periodic force. The effective potential function and the steady-state probability density are derived. The delay time and the strength of its feedback can…

In this paper, we introduce the concept of random time changes in dynamical systems. The sub- ordination principle may be applied to study the long time behavior of the random time systems. We show, under certain assumptions on the class of…

Topic sparsity refers to the observation that individual documents usually focus on several salient topics instead of covering a wide variety of topics, and a real topic adopts a narrow range of terms instead of a wide coverage of the…

Irregularly sampled time series data arise naturally in many application domains including biology, ecology, climate science, astronomy, and health. Such data represent fundamental challenges to many classical models from machine learning…

We report a detailed and systematic study of wave propagation through a stochastic absorbing random medium. Stochastic absorption is modeled by introducing an attenuation constant per unit length $\alpha$ in the free propagation region of…

News consumption behavior is shaped by the coupling between temporal dynamics and content selection. This study proposes a multi-scale temporal-content framework and validates it on two large real-world news datasets, MIND and Adressa.…

Probabilistic models can learn users' preferences from the history of their item adoptions on a social media site, and in turn, recommend new items to users based on learned preferences. However, current models ignore psychological factors…

The recent investigation into the phenomena of refraction and reflection at temporal boundaries, conducted through the lens of spacetime duality, has attracted considerable scholarly interest. This duality unveils insights into the…

This work studies a posteriori error estimates and their use for time-dependent acoustic scattering problems, formulated as a time-dependent boundary integral equation based on a single-layer ansatz. The integral equation is discretized by…

Temporal networks are commonly used to represent dynamical complex systems like social networks, simultaneous firing of neurons, human mobility or public transportation. Their dynamics may evolve on multiple time scales characterising for…

We propose a novel Transformer-based architecture for the task of generative modelling of 3D human motion. Previous work commonly relies on RNN-based models considering shorter forecast horizons reaching a stationary and often implausible…

Softmax attention struggles with long contexts due to structural limitations: the strict sum-to-one constraint forces attention sinks on irrelevant tokens, and probability mass disperses as sequence lengths increase. We tackle these…

Reasoning about graphs evolving over time is a challenging concept in many domains, such as bioinformatics, physics, and social networks. We consider a common case in which edges can be short term interactions (e.g., messaging) or long term…

Human visual perception is a complex, dynamic and fluctuating process. In addition to the incoming visual stimulus, it is affected by many other factors including temporal context, both external and internal to the observer. In this study…

Spatial data exhibits the property that nearby points are correlated. This also holds for learnt representations across layers, but not for commonly used weight initialization methods. Our theoretical analysis quantifies the learning…

Process-oriented theories of cognition must be evaluated against time-ordered observations. Here we present a representative example for data assimilation of the SWIFT model, a dynamical model of the control of spatial fixation position and…

Self-supervised methods have shown remarkable progress in learning high-level semantics and low-level temporal correspondence. Building on these results, we take one step further and explore the possibility of integrating these two features…

Current research on bias in language models (LMs) predominantly focuses on data quality, with significantly less attention paid to model architecture and temporal influences of data. Even more critically, few studies systematically…