Related papers: The Value Functions of Markov Decision Processes

This paper presents a way of solving Markov Decision Processes that combines state abstraction and temporal abstraction. Specifically, we combine state aggregation with the options framework and demonstrate that they work well together and…

In multiple criteria decision aiding, very often the alternatives are compared by means of a value function compatible with the preferences expressed by the Decision Maker. The problem is that, in general, there is a plurality of compatible…

We prove several results concerning classifications, based on successive observations $(X_1,..., X_n)$ of an unknown stationary and ergodic process, for membership in a given class of processes, such as the class of all finite order Markov…

In this short article we present some properties regarding the order and the type of an entire function.

We provide a characterization of the set of real-valued functions that can be the value function of some polynomial game. Specifically, we prove that a function $u : \dR \to \dR$ is the value function of some polynomial game if and only if…

This paper describes the structure of optimal policies for infinite-state Markov Decision Processes with setwise continuous transition probabilities. The action sets may be noncompact. The objective criteria are either the expected total…

We give a concise self-contained presentation of known and new limit theorems for the one-type Markov branching processes with continuous time. The new streamlined proofs are based on what we call, the tail generating function approach. Our…

Markov decision processes (MDPs) are a standard model for sequential decision-making problems and are widely used across many scientific areas, including formal methods and artificial intelligence (AI). MDPs do, however, come with the…

Recently, a class of stochastic processes known as piecewise deterministic Markov processes has been used to define continuous-time Markov chain Monte Carlo algorithms with a number of attractive properties, including compatibility with…

Dynamic programming is a class of algorithms used to compute optimal control policies for Markov decision processes. Dynamic programming is ubiquitous in control theory, and is also the foundation of reinforcement learning. In this paper,…

We consider the down/up crossing property of weighted Markov branching processes. The joint probability distribution of multi crossing numbers of such processes are obtained. In particular, for Markov branching processes, the probability…

Valuation-Based~System can represent knowledge in different domains including probability theory, Dempster-Shafer theory and possibility theory. More recent studies show that the framework of VBS is also appropriate for representing and…

In this paper, we provide strong $L_2$-rates of approximation of the integral-type functionals of Markov processes by integral sums. We improve the method developed in [2]. Under assumptions on the process formulated only in terms of its…

We start from the observation that, anytime two Markov generators share an eigenvalue, the function constructed from the product of the two eigenfunctions associated to this common eigenvalue is a duality function. We push further this…

The paper is concerned with a zero-sum continuous-time stochastic differential game with a dynamics controlled by a Markov process and a terminal payoff. The value function of the original game is estimated using the value function of a…

Value-based methods play a fundamental role in Markov decision processes (MDPs) and reinforcement learning (RL). In this paper, we present a unified control-theoretic framework for analyzing valued-based methods such as value computation…

We develop a qualitative theory of Markov Decision Processes (MDPs) and Partially Observable MDPs that can be used to model sequential decision making tasks when only qualitative information is available. Our approach is based upon an…

The construction presented in this paper can be briefly described as follows: starting from any "finite-dimensional" Markov transition function p_t, on a measurable state space (E,B), we construct a strong Markov process on a certain…

This research aimed to introduce the concept of harmonically m-concave set-valued functions, which is obtained from the combination of two definitions: harmonically m-concave functions and set-valued functions. In this work some properties…

The Markov assumption (MA) is fundamental to the empirical validity of reinforcement learning. In this paper, we propose a novel Forward-Backward Learning procedure to test MA in sequential decision making. The proposed test does not assume…