Related papers: Metastable Distributions of Semi-Markov Processes

In this paper, we consider a class of inhomogeneous semi-Markov processes directly based on intensity processes for marked point processes. We show that this class satisfies the semi-Markov properties defined elsewhere in the literature. We…

We present the convergence rates of synchronous and asynchronous Q-learning for average-reward Markov decision processes, where the absence of contraction poses a fundamental challenge. Existing non-asymptotic results overcome this…

We consider continuous-space, discrete-time Markov chains on $\mathbb{R}^d$, that admit a finite number $N$ of metastable states. Our main motivation for investigating these processes is to analyse random Poincar\'e maps, which describe…

We consider Markov decision processes (MDPs) in which the transition probabilities and rewards belong to an uncertainty set parametrized by a collection of random variables. The probability distributions for these random parameters are…

We consider the problems of parameter estimation for several models of threshold ergodic diffusion processes in the asymptotics of large samples. These models are the direct continuous time analogues of the well-known in time series…

Detecting and resolving violations of temporal constraints in real-time systems is both, time-consuming and resource-intensive, particularly in complex software environments. Measurement-based approaches are widely used during development,…

Using elementary methods, we prove that for a countable Markov chain $P$ of ergodic degree $d > 0$ the rate of convergence towards the stationary distribution is subgeometric of order $n^{-d}$, provided the initial distribution satisfies…

We show that stochastic processes with linear conditional expectations and quadratic conditional variances are Markov, and their transition probabilities are related to a three-parameter family of orthogonal polynomials which generalize the…

The asymptotic behavior of the quantum walk on the line is investigated focusing on the probability distribution of chirality independently of position. The long-time limit of this distribution is shown to exist and to depend on the initial…

In this paper we describe the long time behavior of solutions to quasi-linear parabolic equations with a small parameter at the second order term and the long time behavior of corresponding diffusion processes.

A time-dependent finite-state Markov chain that uses doubly stochastic transition matrices, is considered. Entropic quantities that describe the randomness of the probability vectors, and also the randomness of the discrete paths, are…

This paper is devoted to the estimation of the shift parameter in a semiparametric regression model when the distribution of the observation times is unknown. Hence, we propose to use a stochastic algorithm which takes into account the…

We study Markov processes where the "time" parameter is replaced by paths in a directed graph from an initial vertex to a terminal one. Along each directed path the process is Markov and has the same distribution as the one along any other…

A definition of metastable states applicable to arbitrary finite state Markov processes satisfying detailed balance is discussed. In particular, we identify a crucial condition that distinguishes genuine metastable states from other types…

Random walks are a fundamental model in applied mathematics and are a common example of a Markov chain. The limiting stationary distribution of the Markov chain represents the fraction of the time spent in each state during the stochastic…

We study the limit behaviour of upper and lower bounds on expected time averages in imprecise Markov chains; a generalised type of Markov chain where the local dynamics, traditionally characterised by transition probabilities, are now…

We consider time-inhomogeneous ODEs whose parameters are governed by an underlying ergodic Markov process. When this underlying process is accelerated by a factor $\varepsilon^{-1}$, an averaging phenomenon occurs and the solution of the…

We show that the occurrence of chaotic diffusion in a typical class of time-delayed systems with linear instantaneous and nonlinear delayed term can be well described by an anti-persistent random walk. We numerically investigate the…

We study long run average behavior of generalized semi-Markov processes with both fixed-delay events as well as variable-delay events. We show that allowing two fixed-delay events and one variable-delay event may cause an unstable behavior…

We show that the posterior distribution of parameters in a hidden Markov model with parametric emission distributions and discrete and known state space is asymptotically normal. The main novelty of our proof is that it is based on a…