Related papers: Linear Algebraic Truncation Algorithm with A Poste…
In this paper we consider the problem of obtaining sharp bounds for the performance of temporal difference (TD) methods with linear function approximation for policy evaluation in discounted Markov decision processes. We show that a simple…
To understand the long-run behavior of Markov population models, the computation of the stationary distribution is often a crucial part. We propose a truncation-based approximation that employs a state-space lumping scheme, aggregating…
Determining the induced L2 norm of a linear, parameter-varying (LPV) system is an integral part of many analysis and robust control design procedures. Most prior work has focused on efficiently computing upper bounds for the induced L2…
A typical bottleneck of model predictive control algorithms is the computational burden in order to compute the receding horizon feedback law which is predominantly determined by the length of the prediction horizon. Based on a relaxed…
Time bounded reachability is a fundamental problem in model checking continuous-time Markov chains (CTMCs) and Markov decision processes (CTMDPs) for specifications in continuous stochastic logics. It can be computed by numerically solving…
We consider the dynamics of a linear stochastic approximation algorithm driven by Markovian noise, and derive finite-time bounds on the moments of the error, i.e., deviation of the output of the algorithm from the equilibrium point of an…
We discuss balanced truncation model order reduction for large-scale quadratic-bilinear (QB) systems. Balanced truncation for linear systems mainly involves the computation of the Gramians of the system, namely reachability and…
A highly anticipated use of quantum computers is the simulation of complex quantum systems including molecules and other many-body systems. One promising method involves directly applying a linear combination of unitaries (LCU) to…
Computing the stationary distributions of a continuous-time Markov chain (CTMC) involves solving a set of linear equations. In most cases of interest, the number of equations is infinite or too large, and the equations cannot be solved…
We consider two approaches to balanced truncation of stochastic linear systems, which follow from different generalizations of the reachability Gramian of deterministic systems. Both preserve mean-square asymptotic stability, but only the…
The paper proposes a numerically stable recursive algorithm for the exact computation of the linear-chain conditional random field gradient. It operates as a forward algorithm over the log-domain expectation semiring and has the purpose of…
We propose a model order reduction approach for balanced truncation of linear switched systems. Such systems switch among a finite number of linear subsystems or modes. We compute pairs of controllability and observability Gramians…
We are interested in the analysis of very large continuous-time Markov chains (CTMCs) with many distinct rates. Such models arise naturally in the context of reliability analysis, e.g., of computer network performability analysis, of power…
The aim of this note is to investigate the concentration properties of unbounded functions of geometrically ergodic Markov chains. We derive concentration properties of centered functions with respect to the square of the Lyapunov's…
This paper concerns error bounds for recursive equations subject to Markovian disturbances. Motivating examples abound within the fields of Markov chain Monte Carlo (MCMC) and Reinforcement Learning (RL), and many of these algorithms can be…
In standard balanced truncation model order reduction, the initial condition is typically ignored in the reduction procedure and is assumed to be zero instead. However, such a reduced-order model may be a bad approximation to the full-order…
We consider Markov Decision Processes (MDPs) where the rewards are unknown and may change in an adversarial manner. We provide an algorithm that achieves state-of-the-art regret bound of $O( \sqrt{\tau (\ln|S|+\ln|A|)T}\ln(T))$, where $S$…
Error estimates for the numerical solution of the master equation are presented. Estimates are based on adjoint methods. We find that a good estimate can often be computed without spending computational effort on a dual problem. Estimates…
In recent years probabilistic model checking has become an important area of research because of the diffusion of computational systems of stochastic nature. Despite its great success, standard probabilistic model checking suffers the…
In this paper, we study a Markov chain-based stochastic gradient algorithm in general Hilbert spaces, aiming at approximating the optimal solution of a quadratic loss function. We establish probabilistic upper bounds on its convergence. We…