Related papers: Constructing Strong Markov Processes

Let $(U_n(t))_{t\in\R^d}$ be the empirical process associated to an $\R^d$-valued stationary process $(X_i)_{i\ge 0}$. We give general conditions, which only involve processes $(f(X_i))_{i\ge 0}$ for a restricted class of functions $f$,…

In this paper, we study darning of general symmetric Markov processes by shorting some parts of the state space into singletons. A natural way to construct such processes is via Dirichlet forms restricted to the function space whose members…

This paper draws distinctions among various concepts related to tipping points, robustness, path dependence, and other properties of system dynamics. For each concept a formal definition is provided that utilizes Markov model…

The paper is devoted to the existence of integral functionals $\int_0^\infty f(X(t))\,{\mathrm{d}t}$ for several classes of processes in $\mathbb{R}$ with $d\ge 3$. Some examples such as Brownian motion, fractional Brownian motion, compound…

The theory of ``Markov-up'' processes is being developed. This is a new class of stochastic processes with ``partial'' markovian features; it could also be called ``one-sided Markov''. Such a behavior may be found in the real world and in…

In this paper, we provide strong $L_2$-rates of approximation of the integral-type functionals of Markov processes by integral sums. We improve the method developed in [2]. Under assumptions on the process formulated only in terms of its…

Coarse-graining is a standard method of extracting a simple Markov process from a more complicated one by identifying states. Here we extend coarse-graining to open Markov processes. An "open" Markov process is one where probability can…

We develop a Thermodynamic Formalism for bounded continuous potentials defined on the sequence space $X\equiv E^{\mathbb{N}}$, where $E$ is a general Borel standard space. In particular, we introduce meaningful concepts of entropy and…

A Markov Additive Process is a bi-variate Markov process $(\xi,J)=\big((\xi_t,J_t),t\geq0\big)$ which should be thought of as a multi-type L\'evy process: the second component $J$ is a Markov chain on a finite space $\{1,\ldots,K\}$, and…

Here we propose the Donsker-Varadhan-type compactness conditions and prove the joint large deviation principle for the empirical measure and empirical flow of Markov renewal processes (semi-Markov processes) with a countable state space,…

A Markov decision process can be parameterized by a transition kernel and a reward function. Both play essential roles in the study of reinforcement learning as evidenced by their presence in the Bellman equations. In our inquiry of various…

In this note we re-visit the fundamental question of the strong law of large numbers and central limit theorem for processes in continuous time with conditional stationary and independent increments. For convenience we refer to them as…

Markov decision processes (MDPs) are a standard model for sequential decision-making problems and are widely used across many scientific areas, including formal methods and artificial intelligence (AI). MDPs do, however, come with the…

It is easy to verify the equivalence of the quantum Markov property and the strong additivity of entropy for graded quantum systems as well. However, the structure of Markov states for graded systems is different from that for tensor…

We estimate the distance in total variation between the law of a finite state Markov process at time t, starting from a given initial measure, and its unique invariant measure. We derive upper bounds for the time to reach the equilibrium.…

An infinite system of point particles placed in $\mathds{R}^d$ is studied. Its constituents perform random jumps with mutual repulsion described by a translation-invariant jump kernel and interaction potential, respectively. The pure states…

We consider general Markov chains with discrete time in an arbitrary measurable (phase) space and homogeneous in time. Markov chains are defined by the classical transition function which within the framework of the operator treatment…

Reversibility is a key issue in the interface between computation and physics, and of growing importance as miniaturization progresses towards its physical limits. Most foundational work on reversible computing to date has focussed on…

The aim of this paper is to approximate a finite-state Markov process by another process with fewer states, called herein the approximating process. The approximation problem is formulated using two different methods. The first method,…

We consider an abstract class of infinite-dimensional dynamical systems with inputs. For this class, the significance of noncoercive Lyapunov functions is analyzed. It is shown that the existence of such Lyapunov functions implies…