Related papers: Averaging for some simple constrained Markov proce…
Markov decision processes are useful models of concurrency optimisation problems, but are often intractable for exhaustive verification methods. Recent work has introduced lightweight approximative techniques that sample directly from…
In this paper, we consider a class of slow-fast systems of stochastic partial differential equations where the nonlinearity in the slow equation is not continuous and unbounded. We first provide conditions that ensure the existence of a…
This paper studies continuous-time Markov decision processes under the risk-sensitive average cost criterion. The state space is a finite set, the action space is a Borel space, the cost and transition rates are bounded, and the…
The chaotic nature of fluid flow and the uncertainties in initial conditions limit predictability. Small errors that occur in the initial condition can grow exponentially until they saturate at $\mathcal{O}$(1). Ensemble forecasting…
We introduce a novel framework for efficient sampling from complex, unnormalised target distributions by exploiting multiscale dynamics. Traditional score-based sampling methods either rely on learned approximations of the score function or…
The notion of a successful coupling of Markov processes, based on the idea that both components of the coupled system ``intersect'' in finite time with probability one, is extended to cover situations when the coupling is unnecessarily…
In many learning tasks, certain requirements on the processing of individual data samples should arguably be formalized as strict constraints in the underlying optimization problem, rather than by means of arbitrary penalties. We show that,…
We aim at computing the derivative of the solution to a parametric optimization problem with respect to the involved parameters. For a class broader than that of strongly convex functions, this can be achieved by automatic differentiation…
Irreversible and rejection-free Monte Carlo methods, recently developed in Physics under the name Event-Chain and known in Statistics as Piecewise Deterministic Monte Carlo (PDMC), have proven to produce clear acceleration over standard…
We present an elementary state augmentation method for a class of static risk measure applied to the total cost for both Markov decision processes and stochastic optimal control, such that dynamic programming equations can be derived on the…
A $p$-jump process is a piecewise deterministic Markov process with jumps by a factor of $p$. We prove a limit theorem for such processes on the unit interval. Via duality with respect to probability generating functions, we deduce limiting…
In this paper we investigate the behavior of the bridges of a Markov counting process in several directions. We first characterize convexity(concavity) in time of the mean value in terms of lower (upper) bounds on the so called…
The estimation of absorption time distributions of Markov jump processes is an important task in various branches of statistics and applied probability. While the time-homogeneous case is classic, the time-inhomogeneous case has recently…
For a broad class of point processes, including determinantal point processes, we construct associated marked and conditional ensembles, which allow to study a random configuration in the point process, based on information about a randomly…
Consider a one-sided Markov additive process with an upper and a lower barrier, where each can be either reflecting or terminating. For both defective and non-defective processes and all possible scenarios we identify the corresponding…
This paper is devoted to studying the averaging principle for fast-slow system of rough differential equations driven by mixed fractional Brownian rough path. The fast component is driven by Brownian motion, while the slow component is…
The asymptotic behavior for fully coupled multiscale stochastic systems becomes much complicated when the fast processes do not locate in a compact space. An example is constructed to show that the averaged coefficients may become…
We study continuous time Markov processes on graphs. The notion of frequency is introduced, which serves well as a scaling factor between any Markov time of a continuous time Markov process and that of its jump chain. As an application, we…
For a continuous-time Markov process, we characterize the law of the first jump location when started from an arbitrary initial distribution, in terms of the invariant distribution of an auxiliary Markov process. This could be of interest…
We address the problem of sequentially selecting and observing processes from a given set to find the anomalies among them. The decision-maker observes one process at a time and obtains a noisy binary indicator of whether or not the…