Related papers: Computing hitting times via fluid approximation: a…
We propose a method to compute an approximation of the moments of a discrete-time stochastic polynomial system. We use the Carleman linearization technique to transform this finite-dimensional polynomial system into an infinite-dimensional…
Fluid approximation is a widely used approach for solving two-stage stochastic optimization problems, with broad applications in service system design such as call centers and healthcare operations. However, replacing the underlying random…
We consider the problem of characterising expected hitting times and hitting probabilities for imprecise Markov chains. To this end, we consider three distinct ways in which imprecise Markov chains have been defined in the literature: as…
In this paper, we present a novel iterative Monte Carlo method for approximating the stationary probability of a single state of a positive recurrent Markov chain. We utilize the characterization that the stationary probability of a state…
We develop a stochastic approximation-type algorithm to solve finite state/action, infinite-horizon, risk-aware Markov decision processes. Our algorithm has two loops. The inner loop computes the risk by solving a stochastic saddle-point…
A continuous-time Markov process $X$ can be conditioned to be in a given state at a fixed time $T > 0$ using Doob's $h$-transform. This transform requires the typically intractable transition density of $X$. The effect of the $h$-transform…
We prove a functional limit theorem for Markov chains that, in each step, move up or down by a possibly state dependent constant with probability $1/2$, respectively. The theorem entails that the law of every one-dimensional regular…
Multi-time-scale stochastic approximation is an iterative algorithm for finding the fixed point of a set of $N$ coupled operators given their noisy samples. It has been observed that due to the coupling between the decision variables and…
The study of time-inhomogeneous Markov jump processes is a traditional topic within probability theory that has recently attracted substantial attention in various applications. However, their flexibility also incurs a substantial…
We introduce an approximation strategy for the discounted moments of a stochastic process that can, for a large class of problems, approximate the true moments. These moments appear in pricing formulas of financial products such as bonds…
We consider the problem of approximating optimal in the Minimum Mean Squared Error (MMSE) sense nonlinear filters in a discrete time setting, exploiting properties of stochastically convergent state process approximations. More…
We consider a generalization of the unsplittable maximum two-commodity flow problem on undirected graphs where each commodity $i\in{1,2}$ can be split into a bounded number $k_i$ of equally-sized chunks that can be routed on different…
This paper is dedicated to the investigation of a new numerical method to approximate the optimal stopping problem for a discrete-time continuous state space Markov chain under partial observations. It is based on a two-step discretization…
The law of the iterated logarithm (LIL) for the time-homogeneous Markov process with a unique invariant measure characterizes the almost sure maximum possible fluctuation of time averages around the ergodic limit. Whether a numerical…
We consider sequences $(X_t^N)_{t\geq0}$ of Markov processes in two dimensions whose fluid limit is a stable solution of an ordinary differential equation of the form $\dot{x}_t=b(x_t)$, where $b(x)={\pmatrix{-\mu 0 0 \lambda}}x+\tau(x)$…
We present an approximate analytical expression for the escape rate of time-dependent driven stochastic processes with an absorbing boundary such as the driven leaky integrate-and-fire model for neural spiking. The novel approximation is…
We study the time-bounded reachability problem for continuous-time Markov decision processes (CTMDPs) and games (CTMGs). Existing techniques for this problem use discretisation techniques to break time into discrete intervals, and optimal…
A well-known theorem for an irreducible skip-free chain with absorbing state $d$, under some conditions, is that the hitting (absorbing) time of state $d$ starting from state 0 is distributed as the sum of $d$ independent geometric (or…
We analyze the Gaussian approximation as a method to obtain the first and second moments of a stochastic process described by a master equation. We justify the use of this approximation with ideas coming from van Kampen's expansion approach…
We establish general conditions under which there exists uniform in time convergence between a stochastic process and its approximated system. These standardised conditions consist of a local in time estimate between the original and the…