Related papers: Linear Algebraic Truncation Algorithm with A Poste…
At the working heart of policy iteration algorithms commonly used and studied in the discounted setting of reinforcement learning, the policy evaluation step estimates the value of states with samples from a Markov reward process induced by…
We study the error introduced by entropy regularization in infinite-horizon discrete discounted Markov decision processes. We show that this error decreases exponentially in the inverse regularization strength, both in a weighted…
The theory of imprecise Markov chains has achieved significant progress in recent years. Its applicability, however, is still very much limited, due in large part to the lack of efficient computational methods for calculating…
A delay Lyapunov matrix corresponding to an exponentially stable system of linear time-invariant delay differential equations can be characterized as the solution of a boundary value problem involving a matrix valued delay differential…
An important class of physical systems that are of interest in practice are input-output open quantum systems that can be described by quantum stochastic differential equations and defined on an infinite-dimensional underlying Hilbert…
We analyze structure-preserving model order reduction methods for Ornstein-Uhlenbeck processes and linear S(P)DEs with multiplicative noise based on balanced truncation. For the first time, we include in this study the analysis of non-zero…
Model order reduction is a technique that is used to construct low-order approximations of large-scale dynamical systems. In this paper, we investigate a balancing based model order reduction method for dynamical systems with a linear…
We give computable bounds on the rate of convergence of the transition probabilities to the stationary distribution for a certain class of geometrically ergodic Markov chains. Our results are different from earlier estimates of Meyn and…
In this paper we obtain several informative error bounds on function approximation for the policy evaluation algorithm proposed by Basu et al. when the aim is to find the risk-sensitive cost represented using exponential utility. The main…
Inference in hidden Markov model has been challenging in terms of scalability due to dependencies in the observation data. In this paper, we utilize the inherent memory decay in hidden Markov models, such that the forward and backward…
We derive an approximation error bound that holds simultaneously for a function and all its derivatives up to any prescribed order. The bounds apply to elementary functions, including multivariate polynomials, the exponential function, and…
The problems of optimal recovery of unbounded operators are studied. Optimality means the highest possible accuracy and the minimal amount of discrete information involved. It is established that the truncation method, when certain…
We consider the Bayesian approach to the linear Gaussian inference problem of inferring the initial condition of a linear dynamical system from noisy output measurements taken after the initial time. In practical applications, the large…
We present a method to find an optimal policy with respect to a reward function for a discounted Markov decision process under general linear temporal logic (LTL) specifications. Previous work has either focused on maximizing a cumulative…
We study continuous-time Markov chains on the non-negative integers under mild regularity conditions (in particular, the set of jump vectors is finite and both forward and backward jumps are possible). Based on the so-called flux balance…
Iterative gradient-based optimization algorithms are widely used to solve difficult or large-scale optimization problems. There are many algorithms to choose from, such as gradient descent and its accelerated variants such as Polyak's Heavy…
Markov chains are the de facto finite-state model for stochastic dynamical systems, and Markov decision processes (MDPs) extend Markov chains by incorporating non-deterministic behaviors. Given an MDP and rewards on states, a classical…
This paper presents a novel model order reduction framework tailored for fully nonlinear stochastic dynamics without lifting them to quadratic systems and without using linearization techniques. By directly leveraging structural properties…
We develop a pivot-shifted Carleman linearization framework for quantum algorithms solving quadratic nonlinear ordinary differential equations. By shifting the dynamics by a pivot state prior to Carleman lifting, and combining this with a…
Estimation of the degree of stability and the bounds of solutions to non-autonomous nonlinear systems present major concerns in numerous applied problems. Yet, current techniques are frequently yield overconservative conditions which are…