Related papers: Computing Lower and Upper Hitting Probabilities fo…

In this paper we study reachability verification problems of stochastic discrete-time dynamical systems over the infinite time horizon. The reachability verification of interest in this paper is to certify specified lower and upper bounds…

The concepts of probability, statistics and stochastic theory are being successfully used in structural engineering. Markov Chain modelling is a simple stochastic process model that has found its application in both describing stochastic…

There is a scalability gap between probabilistic and non-probabilistic verification. Probabilistic model checking tools are based either on explicit engines or on (Multi-Terminal) Binary Decision Diagrams. These structures are complemented…

When designing optimal controllers for any system, it is often the case that the true state of the system is unknown to the controller, for example due to noisy measurements or partially observable states. Incomplete state information must…

In this paper we propose a stochastic model predictive control (MPC) algorithm for linear discrete-time systems affected by possibly unbounded additive disturbances and subject to probabilistic constraints. Constraints are treated in…

Hidden Markov models (HMMs) are probabilistic functions of finite Markov chains, or, put in other words, state space models with finite state space. In this paper, we examine subspace estimation methods for HMMs whose output lies a finite…

We consider Markov decision processes (MDPs) in which the transition probabilities and rewards belong to an uncertainty set parametrized by a collection of random variables. The probability distributions for these random parameters are…

Reachability analysis is important for studying optimal control problems and differential games, which are powerful theoretical tools for analyzing and modeling many practical problems in robotics, aircraft control, among other application…

Chance constrained program where one seeks to minimize an objective over decisions which satisfy randomly disturbed constraints with a given probability is computationally intractable. This paper proposes an approximate approach to address…

If the state space of a homogeneous continuous-time Markov chain is too large, making inferences - here limited to determining marginal or limit expectations - becomes computationally infeasible. Fortunately, the state space of such a chain…

In this paper, we consider multi-dimensional maximal cost-bounded reachability probability over continuous-time Markov decision processes (CTMDPs). Our major contributions are as follows. Firstly, we derive an integral characterization…

A new approach for optimal estimation of Markov chains with sparse transition matrices is presented.

Accurately modeling and verifying the correct operation of systems interacting in dynamic environments is challenging. By leveraging parametric uncertainty within the model description, one can relax the requirement to describe exactly the…

We present a general framework for applying learning algorithms and heuristical guidance to the verification of Markov decision processes (MDPs). The primary goal of our techniques is to improve performance by avoiding an exhaustive…

We address the reachability problem for continuous-time stochastic dynamic systems. Our objective is to present a unified framework that characterizes the reachable set of a dynamic system in the presence of both stochastic disturbances and…

Almost sure reachability refers to the property of a stochastic system whereby, from any initial condition, the system state reaches a given target set with probability one. In this paper, we study the problem of certifying almost sure…

In this paper, we address the problem of designing stochastic model predictive control (SMPC) schemes for linear systems affected by unbounded disturbances. The contribution of the paper is rooted in a measured-state initialization…

To verify the correct operation of systems, engineers need to determine the set of configurations of a dynamical model that are able to safely reach a specified configuration under a control law. Unfortunately, constructing models for…

Partially observable Markov decision processes (POMDPs) are a fundamental model for sequential decision-making under uncertainty. However, many verification and synthesis problems for POMDPs are undecidable or intractable. Most prominently,…

In this note, we consider the problem of choosing which nodes of a linear dynamical system should be actuated so that the state transfer from the system's initial condition to a given final state is possible. Assuming a standard complexity…