Related papers: On estimating the memory for finitarily Markovian …

We study the asymptotics of the $k$-regular self-similar fragmentation process. For $\alpha > 0$ and an integer $k \geq 2$, this is the Markov process $(I_t)_{t \geq 0}$ in which each $I_t$ is a union of open subsets of $[0,1)$, and…

In this paper, we obtain a Lamperti type representation for real-valued self-similar Markov processes, killed at their hitting time of zero. Namely, we represent real-valued self-similar Markov processes as time changed multiplicative…

We derive a functional limit theorem for the partial maxima process based on a long memory stationary $\alpha$-stable process. The length of memory in the stable process is parameterized by a certain ergodic-theoretical parameter in an…

We study the long time behaviour of a Markov process evolving in $\mathbb{N}$ and conditioned not to hit 0. Assuming that the process comes back quickly from infinity, we prove that the process admits a unique quasi-stationary distribution…

Predictive constructions are a powerful way of characterizing the probability law of stochastic processes with certain forms of invariance, such as exchangeability or Markov exchangeability. When de Finetti-like representation theorems are…

We develop a qualitative theory of Markov Decision Processes (MDPs) and Partially Observable MDPs that can be used to model sequential decision making tasks when only qualitative information is available. Our approach is based upon an…

We construct a large class of non-Markovian master equations that describe the dynamics of open quantum systems featuring strong memory effects, which relies on a quantum generalization of the concept of classical semi-Markov processes.…

Analysis of non-Markovian systems and memory induced phenomena poses an everlasting challenge for physics. As a paradigmatic example we consider a classical Brownian particle of mass $M$ subjected to an external force and exposed to…

Markov decision processes (MDPs) with rewards are a widespread and well-studied model for systems that make both probabilistic and nondeterministic choices. A fundamental result about MDPs is that their minimal and maximal expected rewards…

Consider a sequence of Markov processes $X^1, X^2,...$ with state space $E$, where $X^N$ has a strong drift to $D \subseteq E$, such that $\Phi(X^N)$ is slow for some appropriate $\Phi: E\to D$. Using the method of martingale problems, we…

We obtain limit theorems for a class of nonlinear discrete-time processes $X(n)$ called the $k$-th order Volterra processes of order $k$. These are moving average $k$-th order polynomial forms: \[…

We consider the problem of minimizing a certainty equivalent of the total or discounted cost over a finite and an infinite time horizon which is generated by a Partially Observable Markov Decision Process (POMDP). The certainty equivalent…

Let $(Z_n)_{n\geq 0}$ be a critical branching process in a random environment defined by a Markov chain $(X_n)_{n\geq 0}$ with values in a finite state space $\mathbb X$. Let $ S_n = \sum_{k=1}^n \ln f_{X_k}'(1)$ be the Markov walk…

Consider a stationary discrete random process with alphabet size d, which is assumed to be the output process of an unknown stationary Hidden Markov Model (HMM). Given the joint probabilities of finite length strings of the process, we are…

Reinforced processes are known to provide a stochastic representation for the quasi-stationary distribution of a given killed Markov process - describing the killed Markov process at fixed time instants. In this paper we shall adapt the…

The aim of this paper is to approximate a finite-state Markov process by another process with fewer states, called herein the approximating process. The approximation problem is formulated using two different methods. The first method,…

Let $X=\{X_n: n\in \mathbb{N}\}$ be a linear process with bounded probability density function $f(x)$. Under certain conditions, we use the kernel estimator \[ \frac{2}{n(n-1)h_n} \sum_{1\le i<j\le n}K\Big(\frac{X_i-X_j}{h_n}\Big) \] to…

Several Markovian process calculi have been proposed in the literature, which differ from each other for various aspects. With regard to the action representation, we distinguish between integrated-time Markovian process calculi, in which…

In the Markov decision process model, policies are usually evaluated by expected cumulative rewards. As this decision criterion is not always suitable, we propose in this paper an algorithm for computing a policy optimal for the quantile…

Semi-Markov processes represent a well known and widely used class of random processes in classical probability theory. Here, we develop an extension of this type of non-Markovian dynamics to the quantum regime. This extension is…