Related papers: On $\mathbb{R}^d$-valued multi-self-similar Markov…
Iteration of randomly chosen quadratic maps defines a Markov process: X_{n+1}=\epsilon_{n+1}X_n(1-X_n), where \epsilon_n are i.i.d. with values in the parameter space [0,4] of quadratic maps F_{\theta}(x)=\theta x(1-x). Its study is of…
The Mittag-Leffler process $X=(X_t)_{t\ge 0}$ is introduced. This Markov process has the property that its marginal random variables $X_t$ are Mittag-Leffler distributed with parameter $e^{-t}$, $t\in [0,\infty)$, and the semigroup…
This paper studies three ways to construct a nonhomogeneous jump Markov process: (i) via a compensator of the random measure of a multivariate point process, (ii) as a minimal solution of the backward Kolmogorov equation, and (iii) as a…
This paper deals with control of partially observable discrete-time stochastic systems. It introduces and studies Markov Decision Processes with Incomplete Information and with semi-uniform Feller transition probabilities. The important…
We consider iterated function systems (finite or countable), together with linear and continuous operators on Hilbert spaces, which enable us to construct Markov-type operators. Under suitable conditions, these Markov-type operators have…
Motivated by queueing systems with heterogeneous parallel servers, we consider a class of structured multi-dimensional Markov processes whose state space can be partitioned into two parts: a finite set of boundary states and a structured…
In order to approximate a continuous time stochastic process by discrete time Markov chains one has several options to embed the Markov chains into continuous time processes. On the one hand there is the Markov embedding, which uses…
In this paper we study the Markov-modulated M/M/$\infty$ queue, with a focus on the correlation structure of the number of jobs in the system. The main results describe the system's asymptotic behavior under a particular scaling of the…
In this paper we consider a discrete scale invariant (DSI) process $\{X(t), t\in {\bf R^+}\}$ with scale $l>1$. We consider to have some fix number of observations in every scale, say $T$, and to get our samples at discrete points…
We study time-changed Markov processes to speed up the convergence of Markov chain Monte Carlo (MCMC) algorithms. The time-changed process is defined by adjusting the speed of time of a base process via a user-chosen, state-dependent…
We consider deterministic Markov decision processes (MDPs) and apply max-plus algebra tools to approximate the value iteration algorithm by a smaller-dimensional iteration based on a representation on dictionaries of value functions. The…
Our main result is the martingale representations for Markov additive processes where the modulator is a Levy process. These processes have three parts: the modulator, the jumps of the ordinate triggered by the modulator, and the…
We treat the class of universal Markov processes on the d-dimensional Euklidean space which do not depend on random. For these, as well as for several subclasses, we prove criteria whether a function f, defined on the positive half-line,…
We are interested by the rate of growth of increasing positive self-similar Markov processes (ipssMp) such that the subordinator associated to it via Lamperti's transformation has infinite mean. We prove that the logarithm of an ipssMp…
Let \xi_t, t\in[0,T], be a strong Markov process with values in a complete separable metric space (X,\rho) and with transition probability function P_{s,t}(x,dy), 0\le s\le t\le T, x\in X. For any h\in[0,T] and a>0, consider the function…
It has been shown by Bertoin and Yor (2002) that the law of positive self-similar Markov processes (pssMps) that only jump downwards and do not hit zero in finite time are uniquely determined by their entire moments for which explicit…
For near-critical, transient Markov chains on the non-negative integers in the Lamperti regime, where the mean drift at $x$ decays as $1/x$ as $x \to \infty$, we quantify degree of transience via existence of moments for conditional return…
In the theory of testing for Markovian processes developed so far, exponentially timed internal actions are not admitted within processes. When present, these actions cannot be abstracted away, because their execution takes a nonzero amount…
We observe the continuous-time Markov Branching Process without high-order moments and allowing Immigration. Limit properties of transition functions and their convergence to invariant measures are investigated. Main mathematical tool is…
We study a time-non-homogeneous Markov process which arose from free probability, and which also appeared in the study of stochastic processes with linear regressions and quadratic conditional variances. Our main result is the explicit…