Related papers: Out of Control: Reducing Probabilistic Models by C…
Markov decision processes (MDPs) are the defacto frame-work for sequential decision making in the presence ofstochastic uncertainty. A classical optimization criterion forMDPs is to maximize the expected discounted-sum pay-off, which…
Statistical model checking (SMC) is a technique for analysis of probabilistic systems that may be (partially) unknown. We present an SMC algorithm for (unbounded) reachability yielding probably approximately correct (PAC) guarantees on the…
Biochemical reaction systems may be viewed as discrete event processes characterized by a number of states and state transitions. These systems may be modeled as state transition systems with transitions representing individual reaction…
We consider the constrained optimal control problem for the gradual-impulsive CTMDP model with the performance criteria being the expected total undiscounted costs (from the running cost and the cost from each time an impulse being…
Handling uncertainty in model predictive control comes with various challenges, especially when considering state constraints under uncertainty. Most methods focus on either the conservative approach of robustly accounting for uncertainty…
Verification of PCTL properties of MDPs with convex uncertainties has been investigated recently by Puggelli et al. However, model checking algorithms typically suffer from state space explosion. In this paper, we address probabilistic…
Probabilistic programs encode stochastic models as ordinary-looking programs with primitives for sampling numbers from predefined distributions and conditioning. Their applications include, among many others, machine learning and modeling…
This article presents a constrained policy optimization approach for the optimal control of systems under nonstationary uncertainties. We introduce an assumption that we call Markov embeddability that allows us to cast the stochastic…
The paper provides a new approach to the determination of a single state value for stochastic output feedback problems using paradigms from Model Predictive Control, particularly the distinction between open-loop and closed-loop control and…
We consider the Chance Constrained Model Predictive Control problem for polynomial systems subject to disturbances. In this problem, we aim at finding optimal control input for given disturbed dynamical system to minimize a given cost…
Discrete-time stochastic systems are an essential modelling tool for many engineering systems. We consider stochastic control systems that are evolving over continuous spaces. For this class of models, methods for the formal verification…
Quantum optimal control experiments and simulations have successfully manipulated the dynamics of systems ranging from atoms to biomolecules. Surprisingly, these collective works indicate that the effort (i.e., the number of algorithmic…
In this work we introduce new approximate similarity relations that are shown to be key for policy (or control) synthesis over general Markov decision processes. The models of interest are discrete-time Markov decision processes, endowed…
Stochastic Model Predictive Control has proved to be an efficient method to plan trajectories in uncertain environments, e.g., for autonomous vehicles. Chance constraints ensure that the probability of collision is bounded by a predefined…
This tutorial paper presents a hands-on perspective on probabilistic model checking with the Storm model checker. Storm is a decade-old model checker that excels in performance and a rich Python-based ecosystem, which makes it easy to…
Robots are increasingly used to carry out critical missions in extreme environments that are hazardous for humans. This requires a high degree of operational autonomy under uncertain conditions, and poses new challenges for assuring the…
Probabilistic Computation Tree Logic (PCTL) is frequently used to formally specify control objectives such as probabilistic reachability and safety. In this work, we focus on model checking PCTL specifications statistically on Markov…
We consider the policy synthesis problem for continuous-state controlled Markov processes evolving in discrete time, when the specification is given as a B\"uchi condition (visit a set of states infinitely often). We decompose computation…
We consider linear model reduction in both the control and state variables for unconstrained linear-quadratic optimal control problems subject to time-varying parabolic PDEs. The first-order optimality condition for a state-space reduced…
Many real-world dynamical systems can be described as State-Space Models (SSMs). In this formulation, each observation is emitted by a latent state, which follows first-order Markovian dynamics. A Probabilistic Deep SSM (ProDSSM)…