Related papers: On characterisation of Markov processes via martin…
There are some positively divisible non-Markovian processes whose transition matrices satisfy the Chapman-Kolmogorov equation. These processes should also satisfy the Kolmogorov consistency conditions, an essential requirement for a process…
Historically time-reversibility of the transitions or processes underpinning Markov chain Monte Carlo methods (MCMC) has played a key r\^ole in their development, while the self-adjointness of associated operators together with the use of…
We present some hardness results on finding the optimal policy for the static formulation of distributionally robust Markov decision processes. We construct problem instances such that when the considered policy class is Markovian and…
The purpose of this paper is to introduce a notion of causality in Markov decision processes based on the probability-raising principle and to analyze its algorithmic properties. The latter includes algorithms for checking cause-effect…
We develop a method for computing policies in Markov decision processes with risk-sensitive measures subject to temporal logic constraints. Specifically, we use a particular risk-sensitive measure from cumulative prospect theory, which has…
We introduce the class of synchronous subsequential relations, a subclass of the synchronous relations which embodies some properties of subsequential relations. If we take relations of this class as forming the possible transitions of an…
Markov decision processes model systems subject to nondeterministic and probabilistic uncertainty. A plethora of verification techniques addresses variations of reachability properties, such as: Is there a scheduler resolving the…
We consider exchangeable Markov multi-state survival processes -- temporal processes taking values over a state-space$\mathcal{S}$ with at least one absorbing failure state $\flat \in \mathcal{S}$ that satisfy natural invariance properties…
We study the complexity of computational problems arising from existence theorems in extremal combinatorics. For some of these problems, a solution is guaranteed to exist based on an iterated application of the Pigeonhole Principle. This…
The dynamics of a Markov process are often specified by its infinitesimal generator or, equivalently, its symbol. This paper contains examples of analytic symbols which do not determine the law of the corresponding Markov process uniquely.…
We assess the descriptive complexity of *bisimilarity* or "equality of behavior" on a family of Markov decision processes over uncountable standard Borel spaces, namely *nondeterministic labelled Markov processes* (NLMP). We show that…
This note provides several recent progresses in the study of long time behavior of Markov processes. The examples presented below are related to other scientific fields as PDE's, physics or biology. The involved mathematical tools as…
Marginal structural models were introduced in order to provide estimates of causal effects from interventions based on observational studies in epidemiological research. The key point is that this can be understood in terms of Girsanov's…
The slow processes of metastable stochastic dynamical systems are difficult to access by direct numerical simulation due the sampling problem. Here, we suggest an approach for modeling the slow parts of Markov processes by approximating the…
Markov chains are a class of probabilistic models that have achieved widespread application in the quantitative sciences. This is in part due to their versatility, but is compounded by the ease with which they can be probed analytically.…
A proof of the continuous martingale convergence theorem is provided. It relies on a classical martingale inequality and the almost sure convergence of a uniformly bounded non-negative super-martingale, after a truncation argument.
Ranking problems, also known as preference learning problems, define a widely spread class of statistical learning problems with many applications, including fraud detection, document ranking, medicine, credit risk screening, image ranking…
The martingale method is used to establish concentration inequalities for a class of dependent random sequences on a countable state space, with the constants in the inequalities expressed in terms of certain mixing coefficients. Along the…
The concern of this paper is the Cauchy problem for the Prandtl equation. This problem is known to be well-posed for analytic data, or for data with monotonicity properties. We prove here that it is linearly ill-posed in Sobolev type…
We study a class of stationary Markov processes with marginal distributions identifiable by moments such that every conditional moment of degree say $m$ is a polynomial of degree at most $m\;\text{.}\;$ We show that then under some…