Related papers: Time series aggregation, disaggregation and long m…
We propose a single chunk model of long-term memory that combines the basic features of the ACT-R theory and the multiple trace memory architecture. The pivot point of the developed theory is a mathematical description of the creation of…
We consider the setting where a collection of time series, modeled as random processes, evolve in a causal manner, and one is interested in learning the graph governing the relationships of these processes. A special case of wide interest…
Many recent long-context and agentic systems address context-length limitations by adding hierarchical memory: they extract atomic units from raw data, build multi-level representatives by grouping and compression, and traverse this…
Aggregation processes with an arbitrary number of conserved quantities are investigated. On the mean-field level, an exact solution for the size distribution is obtained. The asymptotic form of this solution exhibits nontrivial ``double''…
We introduce a recursive algorithm of conveniently general form for estimating the coefficient of a moving average model of order one and obtain convergence results for both correct and misspecified MA(1) models. The algorithm encompasses…
Aggregation of time-series or image data over subsets of the domain is a fundamental task in data science. We show that many known aggregation operations can be interpreted as (double) functors on appropriate (double) categories. Such…
Models for learning probability distributions such as generative models and density estimators behave quite differently from models for learning functions. One example is found in the memorization phenomenon, namely the ultimate convergence…
Self-attention has become an important and widely used neural network component that helped to establish new state-of-the-art results for various applications, such as machine translation and automatic speech recognition (ASR). However, the…
Many random combinatorial objects have a component structure whose joint distribution is equal to that of a process of mutually independent random variables, conditioned on the value of a weighted sum of the variables. It is interesting to…
For many standard models of random structure, first-order logic sentences exhibit a convergence phenomenon on random inputs. The most well-known example is for random graphs with constant edge probability, where the probabilities of…
Methods for the reduction of the complexity of computational problems are presented, as well as their connections to renormalization, scaling, and irreversible statistical mechanics. Several statistically stationary cases are analyzed; for…
The process of data mining produces various patterns from a given data source. The most recognized data mining tasks are the process of discovering frequent itemsets, frequent sequential patterns, frequent sequential rules and frequent…
Existing attention mechanisms are trained to attend to individual items in a collection (the memory) with a predefined, fixed granularity, e.g., a word token or an image grid. We propose area attention: a way to attend to areas in the…
When a system is perturbed by the variation of external parameters, a lag generally develops between the actual state of the system and the equilibrium state corresponding to the current parameter values. We establish a microscopic,…
When analysing time series an important issue is to decide whether the time series is stationary or a random walk. Relaxing these notions, we consider the problem to decide in favor of the I(0)- or I(1)-property. Fixed-sample statistical…
In this paper, we propose and investigate a new neural network architecture called Neural Random Access Machine. It can manipulate and dereference pointers to an external variable-size random-access memory. The model is trained from pure…
This paper introduces the class of ambiguity sparse processes, containing subsets of popular nonstationary time series such as locally stationary, cyclostationary and uniformly modulated processes. The class also contains aggregations of…
We study the effect of stochastic sampling on the estimation of the drift parameter of continuous time AR(1) process. A natural distribution free moment estimator is considered for the drift based on stochastically observed time points. The…
We introduce an alternative to the notion of `fast rate' in Learning Theory, which coincides with the optimal error rate when the given class happens to be convex and regular in some sense. While it is well known that such a rate cannot…
We consider the averaging process on a graph, that is the evolution of a mass distribution undergoing repeated averages along the edges of the graph at the arrival times of independent Poisson processes. We establish cutoff phenomena for…