Related papers: Dynamic Programming for Structured Continuous Mark…
Feature Markov Decision Processes (PhiMDPs) are well-suited for learning agents in general environments. Nevertheless, unstructured (Phi)MDPs are limited to relatively simple environments. Structured MDPs like Dynamic Bayesian Networks…
We investigate piecewise deterministic Markov processes (PDMP), where the deterministic dynamics follows a scalar conservation law and random jumps in the system are characterized by changes in the flux function. We show under which…
We introduce a framework that represents a dynamic program as a family of operators acting on a partially ordered set. We provide an optimality theory based only on order-theoretic assumptions and show how applications across almost all…
We are interested in risk constraints for infinite horizon discrete time Markov decision processes (MDPs). Starting with average reward MDPs, we show that increasing concave stochastic dominance constraints on the empirical distribution of…
Markov decision processes are a ubiquitous formalism for modelling systems with non-deterministic and probabilistic behavior. Verification of these models is subject to the famous state space explosion problem. We alleviate this problem by…
What are the functionals of the reward that can be computed and optimized exactly in Markov Decision Processes?In the finite-horizon, undiscounted setting, Dynamic Programming (DP) can only handle these operations efficiently for certain…
Spatial puzzles composed of rigid objects, flexible strings and holes offer interesting domains for reasoning about spatial entities that are common in the human daily-life's activities. The goal of this work is to investigate the automated…
We formulate a probabilistic Markov property in discrete time under a dynamic risk framework with minimal assumptions. This is useful for recursive solutions to risk-sensitive versions of dynamic optimisation problems such as optimal…
Partially observable Markov decision processes (POMDPs) have recently become popular among many AI researchers because they serve as a natural model for planning under uncertainty. Value iteration is a well-known algorithm for finding…
We consider the problem of computing optimal generalised policies for relational Markov decision processes. We describe an approach combining some of the benefits of purely inductive techniques with those of symbolic dynamic programming…
Leveraging recent work on data-driven methods for constructing a finite state space Markov process from dynamical systems, we address two problems for obtaining further reduced statistical representations. The first problem is to extract…
We provide an overview on how to use the measurable selection techniques to derive the dynamic programming principle for a general stochastic optimal control/stopping problem. By considering its martingale problem formulation on the…
Markov decision processes capture sequential decision making under uncertainty, where an agent must choose actions so as to optimize long term reward. The paper studies efficient reasoning mechanisms for Relational Markov Decision Processes…
Existing decision-theoretic reasoning frameworks such as decision networks use simple data structures and processes. However, decisions are often made based on complex data structures, such as social networks and protein sequences, and rich…
Factored Markov decision processes (MDPs) are a prominent paradigm within the artificial intelligence community for modeling and solving large-scale MDPs whose rewards and dynamics decompose into smaller, loosely interacting components.…
Partially observable Markov decision processes (POMDPs) are a powerful abstraction for tasks that require decision making under uncertainty, and capture a wide range of real world tasks. Today, effective planning approaches exist that…
Symbolic representations have been used successfully in off-line planning algorithms for Markov decision processes. We show that they can also improve the performance of on-line planners. In addition to reducing computation time, symbolic…
This paper presents a new theory, known as robust dynamic pro- gramming, for a class of continuous-time dynamical systems. Different from traditional dynamic programming (DP) methods, this new theory serves as a fundamental tool to analyze…
Decentralized planning in uncertain environments is a complex task generally dealt with by using a decision-theoretic approach, mainly through the framework of Decentralized Partially Observable Markov Decision Processes (DEC-POMDPs).…
In this paper we consider the problem of computing the stationary distribution of nearly completely decomposable Markov processes, a well-established area in the classical theory of Markov processes with broad applications in the design,…