Related papers: Reposition time in probabilistic imperfect memorie…

A continuous-time random walk in the quarter plane with homogeneous transition rates is considered. Given a non-negative reward function on the state space, we are interested in the expected stationary performance. Since a direct derivation…

Temporal-difference (TD) learning is widely regarded as one of the most popular algorithms in reinforcement learning (RL). Despite its widespread use, it has only been recently that researchers have begun to actively study its finite time…

The probability distribution p(l) of an atom to return to a step at distance l from the detachment site, with a random walk in between, is exactly enumerated. In particular, we study the dependence of p(l) on step roughness, presence of…

One of the characteristic features of a stochastic process under resetting is that the probability density converges to a nonequilibrium stationary state (NESS). In addition, the approach to the stationary state exhibits a dynamical phase…

We study the use of Temporal-Difference learning for estimating the structural parameters in dynamic discrete choice models. Our algorithms are based on the conditional choice probability approach but use functional approximations to…

The reciprocal function, 1/x, is important for many real-time algorithms. It is used in a large variety of algorithms from areas ranging from iterative estimation to machine learning. Many of these algorithms are iterative in nature and…

We show a methodology for the computation of the probability of deadline miss for a periodic real-time task scheduled by a resource reservation algorithm. We propose a modelling technique for the system that reduces the computation of such…

We study, in d-dimensions, the random walker with geometrically shrinking step sizes at each hop. We emphasize the integrated quantities such as expectation values, cumulants and moments rather than a direct study of the probability…

This paper focuses on recursive estimation of time varying autoregressive processes in a nonparametric setting. The stability of the model is revisited and uniform results are provided when the time-varying autoregressive parameters belong…

Stochastic approximation algorithm is a useful technique which has been exploited successfully in probability theory and statistics for a long time. The step sizes used in stochastic approximation are generally taken to be deterministic and…

After a quick review of superpositions of OU (supOU) processes, integrated sup\-OU processes and the supOU stochastic volatility model we estimate these processes by using the generalized method of moments (GMM). We show that the GMM…

We look into the problem of stochastic resetting with refractory periods. The model dynamics comprises diffusive and motionless phases. The diffusive phase ends at random time instants, at which the system is reset to a given position --…

We propose autoregressive Bayesian semi-parametric models for waiting times between recurrent events. The aim is two-fold: inference on the effect of possibly time-varying covariates on the gap times and clustering of individuals based on…

For random maps, the expected value of the order (i.e. the period of the sequence of compositional iterates) is approximated asymptotically. It is much smaller than the expected value for the product of the cycle lengths.

This paper describes the probabilistic behaviour of a random Sturmian word. It performs the probabilistic analysis of the recurrence function which can be viewed as a waiting time to discover all the factors of length $n$ of the Sturmian…

We consider a time-varying first-order autoregressive model with irregular innovations, where we assume that the coefficient function is H\"{o}lder continuous. To estimate this function, we use a quasi-maximum likelihood based approach. A…

Processive molecular motors take more-or-less uniformly sized steps, along spatially periodic tracks, mostly forwards but increasingly backwards under loads. Experimentally, the major steps can be resolved clearly within the noise but one…

We present a continuous time generalization of a random walk with complete memory of its history [Phys. Rev. E 70, 045101(R) (2004)] and derive exact expressions for the first four moments of the distribution of displacement when the number…

The idea that memory behavior relies on a gradually-changing internal state has a long history in mathematical psychology. This chapter traces this line of thought from statistical learning theory in the 1950s, through distributed memory…

We consider estimation procedures which are recursive in the sense that each successive estimator is obtained from the previous one by a simple adjustment. We study rate of convergence of recursive estimation procedures for the general…