Related papers: Runtime Monitoring for Markov Decision Processes

In the context of using norms for controlling multi-agent systems, a vitally important question that has not yet been addressed in the literature is the development of mechanisms for monitoring norm compliance under partial action…

A novel data-driven method for formal verification is proposed to study complex systems operating in safety-critical domains. The proposed approach is able to formally verify discrete-time stochastic dynamical systems against temporal logic…

The partial monitoring (PM) framework provides a theoretical formulation of sequential learning problems with incomplete feedback. On each round, a learning agent plays an action while the environment simultaneously chooses an outcome. The…

In this article, we address the problem of risk assessment of stealthy attacks on uncertain control systems. Considering data injection attacks that aim at maximizing impact while remaining undetected, we use the recently proposed…

We consider the problem of minimizing a convex function that is evolving according to unknown and possibly stochastic dynamics, which may depend jointly on time and on the decision variable itself. Such problems abound in the machine…

By adopting a distributional viewpoint on law-invariant convex risk measures, we construct dynamics risk measures (DRMs) at the distributional level. We then apply these DRMs to investigate Markov decision processes, incorporating latent…

Given the inherent non-deterministic nature of machine learning (ML) systems, their behavior in production environments can lead to unforeseen and potentially dangerous outcomes. For a timely detection of unwanted behavior and to prevent…

Partially-observable problems pose a trade-off between reducing costs and gathering information. They can be solved optimally by planning in belief space, but that is often prohibitively expensive. Model-predictive control (MPC) takes the…

This work pioneers regret analysis of risk-sensitive reinforcement learning in partially observable environments with hindsight observation, addressing a gap in theoretical exploration. We introduce a novel formulation that integrates…

Many processes, such as discrete event systems in engineering or population dynamics in biology, evolve in discrete space and continuous time. We consider the problem of optimal decision making in such discrete state and action space…

We present a novel data-driven model predictive control (MPC) approach to control unknown nonlinear systems using only measured input-output data with closed-loop stability guarantees. Our scheme relies on the data-driven system…

In this paper, we investigate the problem of remote estimation of a discrete-time joint Markov process using multiple sensors. Each sensor observes a different component of the joint Markov process, and in each time slot, the monitor…

Many natural and engineered systems can be modeled as discrete state Markov processes. Often, only a subset of states are directly observable. Inferring the conditional probability that a system occupies a particular hidden state, given the…

The number of resident space objects is rising at an alarming rate. Mega-constellations and breakup events are proliferating in most orbital regimes, and safe navigation is becoming increasingly problematic. It is important to be able to…

We consider a continuous-time model for inventory management with Markov modulated non-stationary demands. We introduce active learning by assuming that the state of the world is unobserved and must be inferred by the manager. We also…

A central paradigm behind process semantics based on observability and testing is that the exact moment of occurring of an internal nondeterministic choice is unobservable. It is natural, therefore, for this property to hold when the…

Data generated from dynamical systems with unknown dynamics enable the learning of state observers that are: robust to modeling error, computationally tractable to design, and capable of operating with guaranteed performance. In this paper,…

The security of control systems under sensor attacks is investigated. Redundant observability is introduced, explaining existing security notions including the security index, attack detectability, and observability under attacks.…

In this paper we present a framework for risk-sensitive model predictive control (MPC) of linear systems affected by stochastic multiplicative uncertainty. Our key innovation is to consider a time-consistent, dynamic risk evaluation of the…

We study the problem of system identification for stochastic continuous-time dynamics, based on a single finite-length state trajectory. We present a method for estimating the possibly unstable open-loop matrix by employing properly…