Related papers: Minimal positive Markov realizations
We consider a minimal realization of a rational matrix functions. We perturb the polynomial part and one of the constant matrices from the realization part. We derive explicit computable expressions of backward errors of approximate…
In this paper we consider a control problem for a Partially Observable Piecewise Deterministic Markov Process of the following type: After the jump of the process the controller receives a noisy signal about the state and the aim is to…
We develop the linear programming approach to mean-field games in a general setting. This relaxed control approach allows to prove existence results under weak assumptions, and lends itself well to numerical implementation. We consider…
State-of-the-art probabilistic model checkers perform verification on explicit-state Markov models defined in a high-level programming formalism like the PRISM modeling language. Typically, the low-level models resulting from such…
Low-rank tensor approximation techniques attempt to mitigate the overwhelming complexity of linear algebra tasks arising from high-dimensional applications. In this work, we study the low-rank approximability of solutions to linear systems…
A new algorithm for solving non-homogeneous asymptotically linear and superlinear problems is proposed. The ground state solution of the problem, which in general is obtained as a mini-max of the associated functional, is obtained as the…
We investigate the problem of best policy identification in discounted linear Markov Decision Processes in the fixed confidence setting under a generative model. We first derive an instance-specific lower bound on the expected number of…
We propose a novel randomized linear programming algorithm for approximating the optimal policy of the discounted Markov decision problem. By leveraging the value-policy duality and binary-tree data structures, the algorithm adaptively…
The problem of controlling a finite state Markov chain in the presence of an adversary so as to ensure desired performance levels for a vector of objectives is cast in the framework of Blackwell approachability. Relying on an elementary two…
Positive linear systems on arbitrary time scales are studied. The theory developed in the paper unifies and extends concepts and results known for continuous-time and discrete-time systems. A necessary and sufficient condition for a linear…
The full-dimensional (metric, Euclidean, least squares) multidimensional scaling stress loss function is combined with a quadratic external penalty function term. The trajectory of minimizers of stress for increasing values of the penalty…
We develop tools for investigation of input-to-state stability (ISS) of infinite-dimensional control systems. We show that for certain classes of admissible inputs the existence of an ISS-Lyapunov function implies the input-to-state…
Markov Decision Processes (MDP) is an useful framework to cast optimal sequential decision making problems. Given any MDP the aim is to find the optimal action selection mechanism i.e., the optimal policy. Typically, the optimal policy…
In this paper we discuss various aspects of the problem of determining the minimal dimension of an injective linear representation of a finite semigroup over a field. We outline some general techniques and results, and apply them to…
This paper is devoted to the controllability analysis of a class of linear control systems in a Hilbert space. It is proposed to use the minimum energy controls of a reduced lumped parameter system for solving the infinite dimensional…
Explicit solutions to optimal control problems are rarely obtainable. Of particular interest are the explicit solutions derived for minimax problems, providing a framework to address adversarial conditions and uncertainty. This work…
Generators of Markov processes on a countable state space can be represented as finite or infinite matrices. One key property is that the off-diagonal entries corresponding to jump rates of the Markov process are non-negative. Here we…
We find the minimal number of independent preparations and measurements certifying the dimension of a classical or quantum system limited to $d$ states, optionally reduced to the real subspace. As a dimension certificate, we use the linear…
This paper presents a novel state representation for reward-free Markov decision processes. The idea is to learn, in a self-supervised manner, an embedding space where distances between pairs of embedded states correspond to the minimum…
Many problems in sequential decision making and stochastic control often have natural multiscale structure: sub-tasks are assembled together to accomplish complex goals. Systematically inferring and leveraging hierarchical structure,…