Related papers: Embedded Markov chain approximations in Skorokhod …

Markov models are often used to capture the temporal patterns of sequential data for statistical learning applications. While the Hidden Markov modeling-based learning mechanisms are well studied in literature, we analyze a…

Semi-Markov processes generalize Markov processes by adding temporal memory effects as expressed by a semi-Markov kernel. We recall the path weight for a semi-Markov trajectory and the fact that thermodynamic consistency in equilibrium…

There exists a correlation between geospatial activity temporal patterns and type of land use. A novel self-supervised approach is proposed to stratify landscape based on mobility activity time series. First, the time series signal is…

We study inhomogeneous continuous-time weakly ergodic Markov chains with a finite state space. We introduce the notion of a Markov chain with the regular structure of an infinitesimal matrix and study the sharp upper bounds on the rate of…

A common tool in the practice of Markov Chain Monte Carlo is to use approximating transition kernels to speed up computation when the desired kernel is slow to evaluate or intractable. A limited set of quantitative tools exist to assess the…

We study a class of multi-stage stochastic programs, which incorporate modeling features from Markov decision processes (MDPs). This class includes structured MDPs with continuous action and state spaces. We extend policy graphs to include…

The study of animal behavioural states inferred through hidden Markov models and similar state switching models has seen a significant increase in popularity in recent years. The ability to account for varying levels of behavioural scale…

In this paper, we seek to understand the behavior of dynamical systems that are perturbed by a parameter that changes discretely in time. If we impose certain conditions, we can study certain embedded systems within a hybrid system as…

We propose a random adaptation variant of time-varying distributed averaging dynamics in discrete time. We show that this leads to novel interpretations of fundamental concepts in distributed averaging, opinion dynamics, and distributed…

In the analysis of Markov chains and processes, it is sometimes convenient to replace an unbounded state space with a "truncated" bounded state space. When such a replacement is made, one often wants to know whether the equilibrium behavior…

We give a mathematical framework for Exact Milestoning, a recently introduced algorithm for mapping a continuous time stochastic process into a Markov chain or semi-Markov process that can be efficiently simulated and analyzed. We…

In this paper we consider the Skorokhod embedding problem for target distributions with non-zero mean. In the zero-mean case, uniform integrability provides a natural restriction on the class of embeddings, but this is no longer suitable…

The aim of this paper is to approximate a finite-state Markov process by another process with fewer states, called herein the approximating process. The approximation problem is formulated using two different methods. The first method,…

In this work, we investigate stochastic approximation (SA) with Markovian data and nonlinear updates under constant stepsize $\alpha>0$. Existing work has primarily focused on either i.i.d. data or linear update rules. We take a new…

Max-stable processes are central models for spatial extremes. In this paper, we focus on some space-time max-stable models introduced in Embrechts et al. (2016). The processes considered induce discrete-time Markov chains taking values in…

The estimation of absorption time distributions of Markov jump processes is an important task in various branches of statistics and applied probability. While the time-homogeneous case is classic, the time-inhomogeneous case has recently…

A stochastic Forward-Backward algorithm with a constant step is studied. At each time step, this algorithm involves an independent copy of a couple of random maximal monotone operators. Defining a mean operator as a selection integral, the…

We prove exponential decay of pair correlations for 1D stationary point processes when spacings satisfy a Markov condition, geometric ergodicity, and a condition on exponential moments. The conditions are phrased for stationary sequences of…

Experiments, in particular on biological systems, typically probe lower-dimensional observables which are projections of high-dimensional dynamics. In order to infer consistent models capturing the relevant dynamics of the system, it is…

We consider a stochastic fluid network where the external input processes are compound Poisson with heavy-tailed Weibullian jumps. Our results comprise of large deviations estimates for the buffer content process in the vector-valued…