Related papers: Markov Process Jump Times and Their Cox Constructi…
In this paper, we consider a piecewise deterministic Markov process (PDMP), with known flow and deterministic transition measure, and unknown jump rate $\lambda$. To estimate nonparametrically the jump rate, we first construct an adaptive…
Labeled continuous-time Markov chains (CTMCs) describe processes subject to random timing and partial observability. In applications such as runtime monitoring, we must incorporate past observations. The timing of these observations matters…
The velocity-jump model is a specific type of piecewise deterministic Markov process in which an individual's velocity is constant except at times that form the events of some point process. It represents an interpretable continuous-time…
We describe an exact approach for calculating transition probabilities and waiting times in finite-state discrete-time Markov processes. All the states and the rules for transitions between them must be known in advance. We can then…
In the paper we study continuous time controlled Markov processes using discrete time controlled Markov processes. We consider long run functionals: average reward per unit time or long run risk sensitive functional. We also investigate…
Markov branching systems form a fundamental class of stochastic models that are extensively applied in biology, physics, finance, and other domains. These systems are distinguished by their continuous-time evolution and inherent branching…
In the field of large deviations for stochastic dynamics, the canonical conditioning of a given Markov process with respect to a given time-local trajectory observable over a large time-window has attracted a lot of interest recently. In…
We characterize various forms of positive dependence for a general class of time-inhomogeneous Markov processes called Feller evolution processes (FEPs) and for jump-FEPs. General FEPs can be studied through their time and state-space…
Markov automata combine non-determinism, probabilistic branching, and exponentially distributed delays. This compositional variant of continuous-time Markov decision processes is used in reliability engineering, performance evaluation and…
We consider the Markov random flight $\bold X(t), \; t>0,$ in the three-dimensional Euclidean space $\Bbb R^3$ with constant finite speed $c>0$ and the uniform choice of the initial and each new direction at random time instants that form a…
In this paper we consider a control problem for a Partially Observable Piecewise Deterministic Markov Process of the following type: After the jump of the process the controller receives a noisy signal about the state and the aim is to…
This paper describes the structure of solutions to Kolmogorov's equations for nonhomogeneous jump Markov processes and applications of these results to control of jump stochastic systems. These equations were studied by Feller (1940), who…
Piecewise-deterministic Markov processes (PDMPs) offer a powerful stochastic modeling framework that combines deterministic trajectories with random perturbations at random times. Estimating their local characteristics (particularly the…
Consider a branching Markov process, $X = (X(t), t \ge 0)$, with non-local branching mechanism. Studying the asymptotic behaviour of the moments of X has recently received attention in the literature [6, 7] due to the importance of these…
Before delegating a task to an autonomous system, a human operator may want a guarantee about the behavior of the system. This paper extends previous work on conformal prediction for functional data and conformalized quantile regression to…
In this paper we investigate the behavior of the bridges of a Markov counting process in several directions. We first characterize convexity(concavity) in time of the mean value in terms of lower (upper) bounds on the so called…
We study optimal stopping of Feller-Markov processes to maximise an undiscounted functional consisting of running and terminal rewards. In a finite-time horizon setting, we extend classical results to unbounded rewards. In infinite horizon,…
We present a learning model predictive control (MPC) scheme for chance-constrained Markov jump systems with unknown switching probabilities. Using samples of the underlying Markov chain, ambiguity sets of transition probabilities are…
In this paper we study a family of nonlinear (conditional) expectations that can be understood as a semimartingale with uncertain local characteristics. Here, the differential characteristics are prescribed by a time and path-dependent…
We consider a bivariate process $X_t=(X^1_t,X^2_t)$, which is observed on a finite time interval $[0,T]$ at discrete times $0,\Delta_n,2\Delta_n,....$ Assuming that its two components $X^1$ and $X^2$ have jumps on $[0,T]$, we derive tests…