Related papers: Minimal positive Markov realizations
The paper presents realization theory of discrete-time linear switched systems. A discrete-time linear switched system is a hybrid system, such that the continuous sub-system associated with each discrete state is linear. In this paper we…
In this paper, we present a numerical framework for constructing bounds on stationary performance measures of random walks in the positive orthant using the Markov reward approach. These bounds are established in terms of stationary…
An adaptation of Response Surface Methodology (RSM) when the covariate is of high or infinite dimensional is proposed, providing a tool for black-box optimization in this context. We combine dimension reduction techniques with classical…
Markov decision processes (MDPs) with rewards are a widespread and well-studied model for systems that make both probabilistic and nondeterministic choices. A fundamental result about MDPs is that their minimal and maximal expected rewards…
Continuous-time Markov chains describing interacting processes exhibit a state space that grows exponentially in the number of processes. This state-space explosion renders the computation or storage of the time-marginal distribution, which…
The dynamics of finite dimension open quantum systems is studied with the help of the simplest possible form of projection operators, namely the ones which project only onto one dimensional subspaces. The simplicity of the action of the…
A system is called positive if the set of non-negative states is left invariant by the dynamics. Stability analysis and controller optimization are greatly simplified for such systems. For example, linear Lyapunov functions and storage…
This paper develops a low-nonnegative-rank approximation method to identify the state aggregation structure of a finite-state Markov chain under an assumption that the state space can be mapped into a handful of meta-states. The number of…
This contribution revisits the classical approximate realization problem, which involves determining matrices of a state-space model based on estimates of a truncated series of Markov parameters. A Hankel matrix built up by these Markov…
We present metrics for measuring the similarity of states in a finite Markov decision process (MDP). The formulation of our metrics is based on the notion of bisimulation for MDPs, with an aim towards solving discounted infinite horizon…
Multidimensional systems are becoming increasingly important as they provide a promising tool for estimation, simulation and control, while going beyond the traditional setting of one-dimensional systems. The analysis of multidimensional…
Recently we have presented a hidden variable model of measurements for a qubit where the hidden variable state space dimension is one-half the quantum state manifold dimension. The absence of a short memory (Markov) dynamics is the price…
We consider large-scale Markov decision processes (MDPs) with a risk measure of variability in cost, under the risk-aware MDPs paradigm. Previous studies showed that risk-aware MDPs, based on a minimax approach to handling risk, can be…
The goal of this work is to formally abstract a Markov process evolving in discrete time over a general state space as a finite-state Markov chain, with the objective of precisely approximating its state probability distribution in time,…
We study the computation of lower and upper probabilities of hitting a target set of states for imprecise Markov chains, where transition uncertainty is modelled by a convex set of transition matrices. In the precise case, hitting…
In this paper, we propose an optimization framework to design a network of positive linear systems whose structure switches according to a Markov process. The optimization framework herein proposed allows the network designer to optimize…
We propose a new method for optimistic planning in infinite-horizon discounted Markov decision processes based on the idea of adding regularization to the updates of an otherwise standard approximate value iteration procedure. This…
Nonlinear dimensionality reduction or, equivalently, the approximation of high-dimensional data using a low-dimensional nonlinear manifold is an active area of research. In this paper, we will present a thematically different approach to…
In recent years, a variety of novel measures of dependence have been introduced being capable of characterizing diverse types of directed dependence, hence diverse types of how a number of predictor variables $\mathbf{X} = (X_1, \dots,…
It is well known that zeros and poles of a single-input, single-output system in the transfer function form are the roots of the transfer function's numerator and the denominator polynomial, respectively. However, in the state-space form,…