Related papers: A Regeneration-based a Posteriori Error Bound for …

We develop a framework for the compression of reversible Markov chains with rigorous error control. Given a subset of selected states, we construct reduced dynamics that can be lifted to an approximation of the full dynamics, and we prove…

We present a computer assisted method for generating existence proofs and a posteriori error bounds for solutions to two point boundary value problems (BVPs). All truncation errors are accounted for and, if combined with interval arithmetic…

Computational procedures for the stationary probability distribution, the group inverse of the Markovian kernel and the mean first passage times of an irreducible Markov chain, are developed using perturbations. The derivation of these…

This paper considers the level-increment (LI) truncation approximation of M/G/1-type Markov chains. The LI truncation approximation is usually used to implement Ramaswami's recursion for the stationary distribution in M/G/1-type Markov…

An algorithm for estimating quasi-stationary distribution of finite state space Markov chains has been proven in a previous paper. Now this paper proves a similar algorithm that works for general state space Markov chains under very general…

In this paper we consider the problem of obtaining sharp bounds for the performance of temporal difference (TD) methods with linear function approximation for policy evaluation in discounted Markov decision processes. We show that a simple…

We introduce a unified operator-theoretic framework for analyzing mixing times of finite-state ergodic Markov chains that applies to both reversible and non-reversible dynamics. The central object in our analysis is the projected transition…

In this paper we study a reflected Markov-modulated Brownian motion with a two sided reflection in which the drift, diffusion coefficient and the two boundaries are (jointly) modulated by a finite state space irreducible continuous time…

In this paper, we study a Markov decision process with a non-linear discount function and with a Borel state space. We define a recursive discounted utility, which resembles non-additive utility functions considered in a number of models in…

We revisit the method of Carleman linearization for systems of ordinary differential equations with polynomial right-hand sides. This transformation provides an approximate linearization in a higher-dimensional space through the exact…

We develop the theory of strong stationary duality for diffusion processes on compact intervals. We analytically derive the generator and boundary behavior of the dual process and recover a central tenet of the classical Markov chain theory…

Time bounded reachability is a fundamental problem in model checking continuous-time Markov chains (CTMCs) and Markov decision processes (CTMDPs) for specifications in continuous stochastic logics. It can be computed by numerically solving…

We consider finite element solutions to optimization problems, where the state depends on the possibly constrained control through a linear partial differential equation. Basing upon a reduced and rescaled optimality system, we derive a…

This paper considers an approximation usually used when implementing Ramaswami's recursion for the stationary distribution of the M/G/1-type Markov chain. The approximation is called the level-increment-truncation approximation because it…

In this work, we propose a residual-based a posteriori error estimator for algebraic flux-corrected (AFC) schemes for stationary convection-diffusion equations. A global upper bound is derived for the error in the energy norm for a general…

This article describes an accurate procedure for computing the mean first passage times of a finite irreducible Markov chain and a Markov renewal process. The method is a refinement to the Kohlas, Zeit fur Oper Res, 30,197-207, (1986)…

This paper presents a novel model order reduction framework tailored for fully nonlinear stochastic dynamics without lifting them to quadratic systems and without using linearization techniques. By directly leveraging structural properties…

We study continuous-time Markov chains on the non-negative integers under mild regularity conditions (in particular, the set of jump vectors is finite and both forward and backward jumps are possible). Based on the so-called flux balance…

A common tool in the practice of Markov Chain Monte Carlo is to use approximating transition kernels to speed up computation when the desired kernel is slow to evaluate or intractable. A limited set of quantitative tools exist to assess the…

We propose a model order reduction approach for balanced truncation of linear switched systems. Such systems switch among a finite number of linear subsystems or modes. We compute pairs of controllability and observability Gramians…