Related papers: About Decisiveness of Dynamic Probabilistic Models

Computing optimal conditional reachability probabilities in Markov decision processes (MDPs) is tractable by a reduction to reachability probabilities. Yet, this reduction yields cyclic, challenging MDPs that are often notoriously hard to…

Scaling probabilistic models to large realistic problems and datasets is a key challenge in machine learning. Central to this effort is the development of tractable probabilistic models (TPMs): models whose structure guarantees efficient…

We consider probabilistic automata on infinite words with acceptance defined by safety, reachability, B\"uchi, coB\"uchi, and limit-average conditions. We consider quantitative and qualitative decision problems. We present extensions and…

We present theory and algorithms for the computation of probability-weighted "keep-out" sets to assure probabilistically safe navigation in the presence of multiple rigid body obstacles with stochastic dynamics. Our forward stochastic…

This paper extends the work of Arcidiacono and Miller (2011, 2019) by introducing a novel characterization of finite dependence within dynamic discrete choice models, demonstrating that numerous models display 2-period finite dependence. We…

We consider Piecewise Deterministic Markov Processes (PDMPs) with a finite set of discrete states. In the regime of fast jumps between discrete states, we prove a law of large number and a large deviation principle. In the regime of fast…

This paper deals with control of partially observable discrete-time stochastic systems. It introduces and studies Markov Decision Processes with Incomplete Information and with semi-uniform Feller transition probabilities. The important…

We study the computational complexity of an important property of simple, regular and weighted games, which is decisiveness. We show that this concept can naturally be represented in the context of hypergraph theory, and that decisiveness…

A model predictive control (MPC) scheme for a permanent-magnet synchronous motor (PMSM) is presented. The torque controller optimizes a quadratic cost consisting of control error and machine losses repeatedly, accounting the voltage and…

We study 2-player turn-based perfect-information stochastic games with countably infinite state space. The players aim at maximizing/minimizing the probability of a given event (i.e., measurable set of infinite plays), such as reachability,…

We propose a robust Q-learning algorithm for Markov decision processes under model uncertainty when each state-action pair is associated with a finite ambiguity set of candidate transition kernels. This finite-measure framework enables…

Partially observable Markov decision processes (POMDPs) form a prominent model for uncertainty in sequential decision making. We are interested in constructing algorithms with theoretical guarantees to determine whether the agent has a…

Given Markov chains and Markov decision processes (MDPs) whose transitions are labelled with non-negative integer costs, we study the computational complexity of deciding whether the probability of paths whose accumulated cost satisfies a…

While for deterministic systems, a counterexample to a property can simply be an error trace, counterexamples in probabilistic systems are necessarily more complex. For instance, a set of erroneous traces with a sufficient cumulative…

This paper presents a deep learning based model predictive control algorithm for control affine nonlinear discrete time systems with matched and bounded state dependent uncertainties of unknown structure. Since the structure of…

We consider partially observable Markov decision processes (POMDPs) with limit-average payoff, where a reward value in the interval [0,1] is associated to every transition, and the payoff of an infinite path is the long-run average of the…

The optimization of process economics within the model predictive control (MPC) formulation has given rise to a new control paradigm known as economic MPC (EMPC). Several authors have discussed the closed-loop properties of EMPC-controlled…

Model Predictive Control (MPC) is a widely known control method that has proved to be particularly effective in multivariable and constrained control. Closed-loop stability and recursive feasibility can be guaranteed by employing accurate…

Markov chain Monte Carlo methods have become standard tools in statistics to sample from complex probability measures. Many available techniques rely on discrete-time reversible Markov chains whose transition kernels build up over the…

Imprecise continuous-time Markov chains are a robust type of continuous-time Markov chains that allow for partially specified time-dependent parameters. Computing inferences for them requires the solution of a non-linear differential…