Related papers: On some transformations between positive self--sim…
The objective of this work is to study continuous-time Markov decision processes on a general Borel state space with both impulsive and continuous controls for the infinite-time horizon discounted cost. The continuous-time controlled…
We study the long time behaviour of a Markov process evolving in $\mathbb{N}$ and conditioned not to hit 0. Assuming that the process comes back quickly from infinity, we prove that the process admits a unique quasi-stationary distribution…
The paper addresses the problem of computing maximal conditional expected accumulated rewards until reaching a target state (briefly called maximal conditional expectations) in finite-state Markov decision processes where the condition is…
Piecewise Diffusion Markov Processes (PDifMPs) are valuable for modelling systems where continuous dynamics are interrupted by sudden shifts and/or changes in drift and diffusion. The first-passage time (FPT) in such models plays a central…
This paper deals with control of partially observable discrete-time stochastic systems. It introduces and studies Markov Decision Processes with Incomplete Information and with semi-uniform Feller transition probabilities. The important…
We wish to characterise when a L\'{e}vy process $X_t$ crosses boundaries like $t^\kappa$, $\kappa>0$, in a one or two-sided sense, for small times $t$; thus, we enquire when $\limsup_{t\downarrow 0}|X_t|/t^{\kappa}$, $\limsup_{t\downarrow…
We consider a setting in which the objective is to learn to navigate in a controlled Markov process (CMP) where transition probabilities may abruptly change. For this setting, we propose a performance measure called exploration steps which…
We consider Markov processes in continuous time with state space $\posint^N$ and provide two sufficient conditions and one necessary condition for the existence of moments $E(\|X(t)\|^r)$ of all orders $r \in \nat$ for all $t \geq 0$. The…
A $p$-jump process is a piecewise deterministic Markov process with jumps by a factor of $p$. We prove a limit theorem for such processes on the unit interval. Via duality with respect to probability generating functions, we deduce limiting…
We consider parametric Markov decision processes (pMDPs) that are augmented with unknown probability distributions over parameter values. The problem is to compute the probability to satisfy a temporal logic specification with any concrete…
We show that one can approximate the least fixed point solution for a multivariate system of monotone probabilistic max(min) polynomial equations, referred to as maxPPSs (and minPPSs, respectively), in time polynomial in both the encoding…
We consider a new type of optimal stopping problems where the absorbing boundary moves as the state process X attains new maxima S. More specifically, we set the absorbing boundary as S-b where b is a certain constant. This problem is…
Motion planning under uncertainty for an autonomous system can be formulated as a Markov Decision Process with a continuous state space. In this paper, we propose a novel solution to this decision-theoretic planning problem that directly…
Let \xi_t, t\in[0,T], be a strong Markov process with values in a complete separable metric space (X,\rho) and with transition probability function P_{s,t}(x,dy), 0\le s\le t\le T, x\in X. For any h\in[0,T] and a>0, consider the function…
Markov decision processes (MDP) are finite-state systems with both strategic and probabilistic choices. After fixing a strategy, an MDP produces a sequence of probability distributions over states. The sequence is eventually synchronizing…
We investigate piecewise deterministic Markov processes (PDMP), where the deterministic dynamics follows a scalar conservation law and random jumps in the system are characterized by changes in the flux function. We show under which…
In this paper, we discuss the computation of first passage moments of a time-homogeneous semi-Markov process (SMP) with finite state space to certain of its states that possess the property of universal accessibility (UA). A UA state is one…
In this paper we study a generalized class of Maxwell-Boltzmann equations which in addition to the usual collision term contains a linear deformation term described by a matrix A. This class of equations arises, for instance, from the…
We study one-sided and $\alpha$-correct sequential hypothesis testing for data generated by an ergodic Markov chain. The null hypothesis is that the unknown transition matrix belongs to a prescribed set $P$ of stochastic matrices, and the…
We study invariant boundary conditions for one dimensional discrete Gaussian Markov processes, basic toy models of spatial Markov processes in statistical mechanics. More precisely, we give a decomposition of boundary objects in a non…