Related papers: The Value Functions of Markov Decision Processes
We conduct an investigation of the differentiability and continuity of reward functionals associated to Markovian randomized stopping times. Our focus is mostly on the differentiability, which is a crucial ingredient for a common approach…
We develop notions of integrable functions within the theory of schemic motivic integration.
The minimum value function appearing in Tikhonov regularization technique is very useful in determining the regularization parameter, both theoretically and numerically. In this paper, we discuss the properties of the minimum value…
We construct and study branching Markov processes on the space of finite configurations of the state space of a given standard process, controlled by a branching kernel and a killing one. In particular, we may start with a superprocess,…
We introduce a logic specifically designed to support reasoning about social choice functions. The logic includes operators to capture strategic ability, and operators to capture agent preferences. We establish a correspondence between…
Comparison results for Markov processes w.r.t. function class induced (integral) stochastic orders have a long history. The most general results so far for this problem have been obtained based on the theory of evolution systems on Banach…
The generic identification problem is to decide whether a stochastic process $(X_t)$ is a hidden Markov process and if yes to infer its parameters for all but a subset of parametrizations that form a lower-dimensional subvariety in…
We consider finite horizon Markov decision processes under performance measures that involve both the mean and the variance of the cumulative reward. We show that either randomized or history-based policies can improve performance. We prove…
The paper studies complementary choice functions, i.e. monotonic and consistent choice functions. Such choice functions were introduced and used in the work \cite{RY} for investigation of matchings with complementary contracts. Three…
In this note, we present few examples of Piecewise Deterministic Markov Processes and their long time behavior. They share two important features: they are related to concrete models (in biology, networks, chemistry,. . .) and they are…
Our purpose is to study a particular class of optimal stopping problems for Markov processes. We justify the value function convexity and we deduce that there exists a boundary function such that the smallest optimal stopping time is the…
Recently, Sidford, Wang, Wu and Ye (2018) developed an algorithm combining variance reduction techniques with value iteration to solve discounted Markov decision processes. This algorithm has a sublinear complexity when the discount factor…
In the literature on spatial point processes, there is an emerging challenge in studying marked point processes with points being labelled by functions. In this paper, we focus on point processes living on linear networks and, from distinct…
Partially observable Markov decision processes (POMDPs) have recently become popular among many AI researchers because they serve as a natural model for planning under uncertainty. Value iteration is a well-known algorithm for finding…
General birth-and-death as well as hopping stochastic dynamics of infinite particle systems in the continuum are considered. We derive corresponding evolution equations for correlation functions and generating functionals. General…
Markov decision processes are typically used for sequential decision making under uncertainty. For many aspects however, ranging from constrained or safe specifications to various kinds of temporal (non-Markovian) dependencies in task and…
Given a strictly positive measure, we characterize inner semicontinuous solid convex-valued mappings for which continuous functions which are selections almost everywhere are selections. This class contains continuous mappings as well as…
Markov Decision Processes (MDPs) have been used to formulate many decision-making problems in science and engineering. The objective is to synthesize the best decision (action selection) policies to maximize expected rewards (or minimize…
We consider random processes that are history-dependent, in the sense that the distribution of the next step of the process at any time depends upon the entire past history of the process. In general, therefore, the Markov property cannot…
Integral properties of multifunctions determined by vector valued functions are presented. Such multifunctions quite often serve as examples and counterexamples. In particular it can be observed that the properties of being integrable in…