Related papers: A Notation for Markov Decision Processes
Markov decision processes (MDP) are useful to model concurrent process optimisation problems, but verifying them with numerical methods is often intractable. Existing approximative approaches do not scale well and are limited to memoryless…
The nominal transition systems (NTSs) of Parrow et al. describe the operational semantics of nominal process calculi. We study NTSs in terms of the nominal residual transition systems (NRTSs) that we introduce. We provide rule formats for…
In this paper we study resolutions which arise as iterated mapping cones.
We consider random processes that are history-dependent, in the sense that the distribution of the next step of the process at any time depends upon the entire past history of the process. In general, therefore, the Markov property cannot…
Lecture notes (in French) of a master 2 level course in applied mathematics. Contents: Part I. Markov chains on a countable space. 1. Examples 2. Summary of basic properties. 3. Spectral theory and speed of convergence. 4. Lyapunov…
Opacity is a generic security property, that has been defined on (non probabilistic) transition systems and later on Markov chains with labels. For a secret predicate, given as a subset of runs, and a function describing the view of an…
Markov models are widely used to describe processes of stochastic dynamics. Here, we show that Markov models are a natural consequence of the dynamical principle of Maximum Caliber. First, we show that when there are different possible…
This note provides several recent progresses in the study of long time behavior of Markov processes. The examples presented below are related to other scientific fields as PDE's, physics or biology. The involved mathematical tools as…
Markov decision processes are a ubiquitous formalism for modelling systems with non-deterministic and probabilistic behavior. Verification of these models is subject to the famous state space explosion problem. We alleviate this problem by…
A stochastic calculus is given for processes described by stochastic integrals with respect to fractional Brownian motions and Rosenblatt processes somewhat analogous to the stochastic calculus for It\^{o} processes. These processes for…
This paper is dedicated to the numerical study of the optimization of an industrial launcher integration process. It is an original case of inventory-production system where a calendar plays a crucial role. The process is modeled using the…
We consider Markov decision processes (MDPs) which are a standard model for probabilistic systems. We focus on qualitative properties for MDPs that can express that desired behaviors of the system arise almost-surely (with probability 1) or…
This note provides a tool to infer moderate deviations principles for specific random variables from deviations principles for their Hubbard-Stratonovich transforms.
In this article almost semi-continuous processes with stationary independent increments on a finite irreducible Markov chain are considered. For these processes the components of matrix factorization identity are concretely defined. On the…
In this note we give a wellfoundedness proof of a computable notation system for first-order reflection.
Several Markovian process calculi have been proposed in the literature, which differ from each other for various aspects. With regard to the action representation, we distinguish between integrated-time Markovian process calculi, in which…
In this work we present a modified neural network model which is capable to simulate Markov Chains. We show how to express and train such a network, how to ensure given statistical properties reflected in the training data and we…
We study a class of multi-stage stochastic programs, which incorporate modeling features from Markov decision processes (MDPs). This class includes structured MDPs with continuous action and state spaces. We extend policy graphs to include…
This paper deals with the question of how to most effectively conduct experiments in Partially Observed Markov Decision Processes so as to provide data that is most informative about a parameter of interest. Methods from Markov decision…
We use the abstract method of (local) martingale problems in order to give criteria for convergence of stochastic processes. Extending previous notions, the formulation we use is neither restricted to Markov processes (or semimartingales),…