相关论文: Stochastic Parroting in Temporal Attention -- Regu…
To make informed decisions in natural environments that change over time, humans must update their beliefs as new observations are gathered. Studies exploring human inference as a dynamical process that unfolds in time have focused on…
We introduce a novel segmental-attention model for automatic speech recognition. We restrict the decoder attention to segments to avoid quadratic runtime of global attention, better generalize to long sequences, and eventually enable…
Many different approaches for estimating the Interaction Quality (IQ) of Spoken Dialogue Systems have been investigated. While dialogues clearly have a sequential nature, statistical classification approaches designed for sequential…
Enhancement of the predictive power and robustness of nonlinear population dynamics models allows ecologists to make more reliable forecasts about species' long term survival. However, the limited availability of detailed ecological data,…
Visual attention mechanisms have proven to be integrally important constituent components of many modern deep neural architectures. They provide an efficient and effective way to utilize visual information selectively, which has shown to be…
We use the two-time scale subordination in order to describe dynamical processes in continuous media with a long-term memory. Our consideration touches two physical examples in detail. First we study a temporal evolution of the species…
We develop a framework to track the structure of temporal networks with a signal processing approach. The method is based on the duality between networks and signals using a multidimensional scaling technique. This enables a study of the…
We study the long time behavior (homogenization) of a diffusion in random medium with time and space dependent coefficients. The diffusion coefficient may degenerate. In Stochastic Process. Appl. (2007) (to appear), an invariance principle…
High-dimensional multivariate spatial-temporal data arise frequently in a wide range of applications; however, there are relatively few statistical methods that can simultaneously deal with spatial, temporal and variable-wise dependencies…
We propose a novel framework for learning time-varying graphs from spatiotemporal measurements. Given an appropriate prior on the temporal behavior of signals, our proposed method can estimate time-varying graphs from a small number of…
The temporal statistics exhibited by written correspondence appear to be media dependent, with features which have so far proven difficult to characterize. We explain the origin of these difficulties by disentangling the role of spontaneous…
The growing interest in complex decision-making and language modeling problems highlights the importance of sample-efficient learning over very long horizons. This work takes a step in this direction by investigating contextual linear…
Recent analyses combining advanced theoretical techniques and high-quality data from thousands of simultaneously recorded neurons provide strong support for the hypothesis that neural dynamics operate near the edge of instability across…
In this paper we introduce the concept of random time changes in dynamical systems. The subordination principle may be applied to study the long time behavior of the random time systems. We show, under certain assumptions on the class of…
We study the correlations of time delays in a model of chaotic resonance scattering based on the random matrix approach. Analytical formulae which are valid for arbitrary number of open channels and arbitrary coupling strength between…
The capacity to filter out irrelevant information from our environment is critical to efficient processing. Yet, during development, when building a knowledge base of the world is occurring, the ability to selectively allocate attentional…
Transformers have proven highly effective across modalities, but standard softmax attention scales quadratically with sequence length, limiting long context modeling. Linear attention mitigates this by approximating attention with kernel…
Pathways of diffusion observed in real-world systems often require stochastic processes going beyond first-order Markov models, as implicitly assumed in network theory. In this work, we focus on second-order Markov models, and derive an…
In this paper, we present the Sub-Adjacent Transformer with a novel attention mechanism for unsupervised time series anomaly detection. Unlike previous approaches that rely on all the points within some neighborhood for time point…
We consider the statistics of time delay in a chaotic cavity having $M$ open channels, in the absence of time-reversal invariance. In the random matrix theory approach, we compute the average value of polynomial functions of the time delay…