Related papers: A Notation for Markov Decision Processes
This work provides a novel interpretation of Markov Decision Processes (MDP) from the online optimization viewpoint. In such an online optimization context, the policy of the MDP is viewed as the decision variable while the corresponding…
Markov matrices have an important role in the filed of stochastic processes. In this paper, we will show and prove a series of conclusions on Markov matrices and transformations rather than pay attention to stochastic processes although…
We define a Markov process on the set of countable graphs with spins. Transitions are local substitutions in the graph. It is proved that the scaling macrodimension is an invariant of such dynamics.
In the development of stochastic integration and the theory of semimartingales, Markov processes have been a constant source of inspiration. Despite this historical interweaving, it turned out that semimartingales should be considered the…
We present a list of algebraic, combinatorial, and analytic mechanisms that give rise to determinantal point processes.
Relational Markov Decision Processes are a useful abstraction for complex reinforcement learning problems and stochastic planning problems. Recent work developed representation schemes and algorithms for planning in such problems using the…
A hidden Markov process is a well known concept in information theory and is used for a vast range of applications such as speech recognition and error correction. We bridge between two disciplines, experimental physics and advanced…
The generic identification problem is to decide whether a stochastic process $(X_t)$ is a hidden Markov process and if yes to infer its parameters for all but a subset of parametrizations that form a lower-dimensional subvariety in…
This paper studies average-cost Markov decision processes with semi-uniform Feller transition probabilities. This class of MDPs was recently introduced by the authors to study MDPs with incomplete information. This paper studies the…
We introduce a framework for approximate analysis of Markov decision processes (MDP) with bounded-, unbounded-, and infinite-horizon properties. The main idea is to identify a "core" of an MDP, i.e., a subsystem where we provably remain…
Using appropriate notation systems for proofs, cut-reduction can often be rendered feasible on these notations, and explicit bounds can be given. Developing a suitable notation system for Bounded Arithmetic, and applying these bounds, all…
This paper is simply a collection of process diagrams for further use and reference. These are diagrams about different approaches to research.
This paper gives an overview of notations used in multiway array processing. We redefine the vectorization and matricization operators to comply with some properties of the Kronecker product. The tensor product and Kronecker product are…
A known property of conditional expectation is extended to the framework of Markov kernels. Its meaning in terms of densities is provided. Some examples located in the field of clinical diagnosis are presented to delimit the main result of…
In this preliminary note, we will illustrate our ideas on automated mechanisms for termination and non-termination reasoning.
For Cox processes we construct a Markov process with increasing paths to couple the condensations of the Cox process in a monotone way. A similar procedure procedure yields an analogue Markov process for the P\'olya sum process. Moreover,…
Recently two approximate Newton methods were proposed for the optimisation of Markov Decision Processes. While these methods were shown to have desirable properties, such as a guarantee that the preconditioner is negative-semidefinite when…
This paper aims to establish a central limit theorem for Markov processes conditioned not to be absorbed under a very general assumption on quasi-stationarity for the underlying process. To do so, a central limit theorem has been…
We introduce multi-kangaroo Markov processes and provide a general procedure for evaluating a certain type of stochastic functionals. We calculate analytically the large deviation properties. Applications include zero-crossing statistics…
Partially observable Markov decision processes have been widely used to provide models for real-world decision making problems. In this paper, we will provide a method in which a slightly different version of them called Mixed observability…