相关论文: Computing hitting times via fluid approximation: a…
Fluid approximations have seen great success in approximating the macro-scale behaviour of Markov systems with a large number of discrete states. However, these methods rely on the continuous-time Markov chain (CTMC) having a particular…
We consider the problem of bounding mean first passage times for a class of continuous-time Markov chains that captures stochastic interactions between groups of identical agents. The quantitative analysis of such probabilistic population…
Hybrid systems, and Piecewise Deterministic Markov Processes in particular, are widely used to model and numerically study systems exhibiting multiple time scales in biochemical reaction kinetics and related areas. In this paper an almost…
We propose a solution to a time-varying variant of Markov Decision Processes which can be used to address decision-theoretic planning problems for autonomous systems operating in unstructured outdoor environments. We explore the time…
In this paper we investigate a potential use of fluid approximation techniques in the context of stochastic model checking of CSL formulae. We focus on properties describing the behaviour of a single agent in a (large) population of agents,…
We present the conditions under which the time-optimal control problem for a nonlinear non-autonomous linearizable system can be solved by the method of successive approximations, at each step of which a power Markov moment min-problem is…
In this paper we propose a new method for approximating the nonstationary moment dynamics of one dimensional Markovian birth-death processes. By expanding the transition probabilities of the Markov process in terms of Poisson-Charlier…
This paper presents a numerical method to calculate the value function for a general discounted impulse control problem for piecewise deterministic Markov processes. Our approach is based on a quantization technique for the underlying…
The aim of this paper is to approximate a finite-state Markov process by another process with fewer states, called herein the approximating process. The approximation problem is formulated using two different methods. The first method,…
In this work we consider a simple random walk embedded in a generic branched structure and we find a close-form formula to calculate the hitting time $H\left(i,f\right)$ between two arbitrary nodes $i$ and $j$. We then use this formula to…
We study an approximation method of stationary characters of a two-dimensional Markov chain via the Stein method. For this purpose, innovative methods are developed to estimate the moments of the Markov chain, as well as the solution to the…
In the first part of this paper we introduced an algorithm that uses reachable set approximation to approximate the minimum time function of linear control problems. To illustrate the error estimates and to demonstrate differences to other…
We study the Multiple Cluster Scheduling problem and the Multiple Strip Packing problem. For both problems, there is no algorithm with approximation ratio better than $2$ unless $P = NP$. In this paper, we present an algorithm with…
We propose a numerical method to approximate the value function for the optimal stopping problem of a piecewise deterministic Markov process (PDMP). Our approach is based on quantization of the post jump location---inter-arrival time Markov…
Random events in space and time often exhibit a locally dependent structure. When the events are very rare and dependent structure is not too complicated, various studies in the literature have shown that Poisson and compound Poisson…
Motivated by a problem in the theory of randomized search heuristics, we give a very precise analysis for the coupon collector problem where the collector starts with a random set of coupons (chosen uniformly from all sets). We show that…
We consider the problem of estimating the asymptotic variance of a function defined on a Markov chain, an important step for statistical inference of the stationary mean. We design a novel recursive estimator that requires $O(1)$…
Reachability analysis of hybrid systems has been used as a safety verification tool to assess offline whether the state of a system is capable of remaining within a designated safe region for a given time horizon. Although it has been…
We present a dynamic programming-based solution to a stochastic optimal control problem up to a hitting time for a discrete-time Markov control process. Firstly, we determine an optimal control policy to steer the process toward a compact…
In this paper we show how to generalize the quantum approximate counting technique developed by Brassard, H{\o}yer and Tapp [ICALP 1998] to a more general setting: estimating the number of marked states of a Markov chain (a Markov chain can…