Related papers: Stochastic chains with memory of variable length

How to model distribution of sequential data, including but not limited to speech and human motions, is an important ongoing research problem. It has been demonstrated that model capacity can be significantly enhanced by introducing…

We describe a new algorithm for the perfect simulation of variable length Markov chains and random systems with perfect connections. This algorithm, which generalizes Propp and Wilson's simulation scheme, is based on the idea of coupling…

A stochastic model, the product of a circulant matrix and a random normal vector, is shown to produce an evolutive long memory time series with a power law spectral density. The distribution of the time series, a beta location scale family…

We introduce a class of stochastic integer sequences. In these sequences, every element is a sum of two previous elements, at least one of which is chosen randomly. The interplay between randomness and memory underlying these sequences…

This talk will review a little over a decade's research on applying certain stochastic models to biological sequence analysis. The models themselves have a longer history, going back over 30 years, although many novel variants have arisen…

This work theoretically studies stochastic neural networks, a main type of neural network in use. We prove that as the width of an optimized stochastic neural network tends to infinity, its predictive variance on the training set decreases…

A theory of additive Markov chains with long-range memory, proposed earlier in Phys. Rev. E 68, 06117 (2003), is developed and used to describe statistical properties of long-range correlated systems. The convenient characteristics of such…

From a continuous-time long memory stochastic process, a discrete-time randomly sampled one is drawn. We investigate the second-order properties of this process and establish some time-and frequency-domain asymptotic results. We mainly…

Stochastic approximation algorithm is a useful technique which has been exploited successfully in probability theory and statistics for a long time. The step sizes used in stochastic approximation are generally taken to be deterministic and…

Stochastic memoization is a higher-order construct of probabilistic programming languages that is key in Bayesian nonparametrics, a modular approach that allows us to extend models beyond their parametric limitations and compose them in an…

We propose a new approach to apply the chaining technique in conjunction with information-theoretic measures to bound the generalization error of machine learning algorithms. Different from the deterministic chaining approach based on…

The mathematical model of a linear system with the short memory about own stochastic behavior is proposed. It is assumed that the system is under a continual influence of independent stochastic impulses. In a short memory approximation the…

A new approach to describing correlation properties of complex dynamic systems with long-range memory based on a concept of additive Markov chains (Phys. Rev. E 68, 061107 (2003)) is developed. An equation connecting a memory function of…

Infinite random sequences of letters can be viewed as stochastic chains or as strings produced by a source, in the sense of information theory. The relationship between Variable Length Markov Chains (VLMC) and probabilistic dynamical…

We establish sufficient conditions for perfect simulation of chains of infinite order on a countable alphabet. The new assumption, localized continuity, is formalized with the help of the notion of context trees, and includes the…

Encoding temporal information from the recent past as spatially distributed activations is essential in order for the entire recent past to be simultaneously accessible. Any biological or synthetic agent that relies on the past to…

Models for categorical sequences typically assume exchangeable or first-order dependent sequence elements. These are common assumptions, for example, in models of computer malware traces and protein sequences. Although such simplifying…

A stochastic model of short-term verbal memory is proposed, in which the psychological state of the subject is encoded as the instantaneous position of a particle diffusing over a semantic graph with a probabilistic structure. The model is…

In the recent publication (arxiv:2007.08063v2 [cs.LG]) a fast prediction algorithm for a single recurrent network (RN) was suggested. In this manuscript we generalize this approach to a chain of RNs and show that it can be implemented in…

This paper presents the design of an associative memory with feedback that is capable of on-line temporal sequence learning. A framework for on-line sequence learning has been proposed, and different sequence learning models have been…