Related papers: P-adic Markov process and the problem of the first…
We study a time-non-homogeneous Markov process which arose from free probability, and which also appeared in the study of stochastic processes with linear regressions and quadratic conditional variances. Our main result is the explicit…
Semi-Markov processes are Markovian processes in which the firing time of the transitions is modelled by probabilistic distributions over positive reals interpreted as the probability of firing a transition at a certain moment in time. In…
A systematic exposition of scale functions is given for positive self-similar Markov processes (pssMp) with one-sided jumps. The scale functions express as convolution series of the usual scale functions associated with spectrally one-sided…
Scaled type Markov renewal processes generalize classical renewal processes: renewal times come from a one parameter family of probability laws and the sequence of the parameters is the trajectory of an ergodic Markov chain. Our primary…
The dynamics in games involving multiple players, who adaptively learn from their past experience, is not yet well understood. We analyzed a class of stochastic games with Markov strategies in which players choose their actions…
In this paper, we provide a novel algorithm for solving planning and learning problems of Markov decision processes. The proposed algorithm follows a policy iteration-type update by using a rank-one approximation of the transition…
The classical inverse first passage time problem asks whether, for a Brownian motion $(B_t)_{t\geq 0}$ and a positive random variable $\xi$, there exists a barrier $b:\mathbb{R}_+\to\mathbb{R}$ such that $\mathbb{P}\{B_s>b(s), 0\leq s \leq…
We consider perturbations of positive recurrent Markov modulated fluid models. In addition to the infinitesimal generator of the phases, we also perturb the rate matrix, and analyze the effect of those perturbations on the matrix of first…
This paper surveys the analysis of parametric Markov models whose transitions are labelled with functions over a finite set of parameters. These models are symbolic representations of uncountable many concrete probabilistic models, each…
We consider a Markovian jumping process with two absorbing barriers, for which the waiting-time distribution involves a position-dependent coefficient. We solve the Fokker-Planck equation with boundary conditions and calculate the mean…
This article treats both discrete time and continuous time stopping problems for general Markov processes on the real line with general linear costs. Using an auxiliary function of maximum representation type, conditions are given to…
For a positive self-similar Markov process, X, we construct a local time for the random set, $\Theta$, of times where the process reaches its past supremum. Using this local time we describe an exit system for the excursions of X out of its…
In this paper we introduce and solve a class of optimal stopping problems of recursive type. In particular, the stopping payoff depends directly on the value function of the problem itself. In a multi-dimensional Markovian setting we show…
We study and provide efficient algorithms for multi-objective model checking problems for Markov Decision Processes (MDPs). Given an MDP, M, and given multiple linear-time (\omega -regular or LTL) properties \varphi\_i, and probabilities…
We characterize the value function and the optimal stopping time for a large class of optimal stopping problems where the underlying process to be stopped is a fairly general Markov process. The main result is inspired by recent findings…
In this paper, we investigate stochastic versions of the Hopf-Lax formula which are based on compositions of the Hopf-Lax operator with the transition kernel of a L\'evy process taking values in a separable Banach space. We show that,…
For a given Markov process $X$ and survival function $\overline{H}$ on $\mathbb{R}^+$, the inverse first-passage time problem (IFPT) is to find a barrier function $b:\mathbb{R}^+\to[-\infty,+\infty]$ such that the survival function of the…
This short paper presents two open problems on the widely used Polyak's Heavy-Ball algorithm. The first problem is the method's ability to exactly \textit{accelerate} in dimension one exactly. The second question regards the behavior of the…
We consider a run-and-tumble particle on a half-line with an absorbing target at the origin. The particle has an internal velocity state that switches between two opposite values at Poisson-distributed times. The position of the particle…
We uncover a duality between relaxation and first passage processes in ergodic reversible Markovian dynamics in both discrete and continuous state-space. The duality exists in the form of a spectral interlacing -- the respective time scales…