Related papers: Markovian Testing Equivalence and Exponentially Ti…
We investigate the decidability of model-checking logics of time, knowledge and probability, with respect to two epistemic semantics: the clock and synchronous perfect recall semantics in partially observed discrete-time Markov chains.…
This paper is concerned with ergodic properties of inhomogeneous Markov processes. Since the transition probabilities depend on initial times, the existing methods to obtain invariant measures for homogeneous Markov processes are not…
In the hidden Markov process, there is a possibility that two different transition matrices for hidden and observed variables yield the same stochastic behavior for the observed variables. Since such two transition matrices cannot be…
We continue study of conformal testing in binary model situations. In this note we consider Markov alternatives to the null hypothesis of exchangeability. We propose two new classes of conformal test martingales; one class is statistically…
We assess non-Markovianity of a quantum open-system dynamics through the violation of temporal bell-like inequalities in a controllable Nuclear Magnetic Resonance system. We investigate experimentally the connections between the violation…
We characterize a class of Markovian dynamics using the concept of divisible dynamical map. Moreover we provide a family of criteria which can distinguish Markovian and non-Markovian dynamics. These Markovianity criteria are based on a…
Continuous measurements are central to quantum control and sensing, yet lack a model-independent operational description that can be applied to arbitrary non-Markovian processes without specifying a microscopic measurement model. Existing…
A Markov Additive Process is a bi-variate Markov process $(\xi,J)=\big((\xi_t,J_t),t\geq0\big)$ which should be thought of as a multi-type L\'evy process: the second component $J$ is a Markov chain on a finite space $\{1,\ldots,K\}$, and…
In this brief paper we find computable exponential convergence rates for a large class of stochastically ordered Markov processes. We extend the result of Lund, Meyn, and Tweedie (1996), who found exponential convergence rates for…
The interaction between an open quantum system and its environment induces generally memory effects generated by the fact that the response of the system to the environment is not instantaneous. Different physical reasons can be at the…
The aim of this paper is to propose a methodology for testing general hypothesis in a Markovian setting with random sampling. A discrete Markov chain X is observed at random time intervals $\tau$ k, assumed to be iid with unknown…
Reward is the driving force for reinforcement-learning agents. This paper is dedicated to understanding the expressivity of reward as a way to capture tasks that we would want an agent to perform. We frame this study around three new…
We consider overdamped physical systems evolving under a feedback-controlled fluctuating potential and in contact with a thermal bath at temperature $T$. A Markovian description of the dynamics, which keeps only the last value of the…
We investigate the role of coherence and Markovianity in finding an answer to the question whether the outcomes of a projectively measured quantum stochastic process are compatible with a classical stochastic process. For this purpose we…
. Markov chains in time, such as simple random walks, are at the heart of probability. In space, due to the absence of an obvious definition of past and future, a range of definitions of Markovianity have been proposed. In this paper, after…
For a general attractive Probabilistic Cellular Automata on S Z d , we prove that the (time-) convergence towards equilibrium of this Markovian parallel dynamics, exponentially fast in the uniform norm, is equivalent to a condition (A).…
We present a probabilistic model for stochastic iterative algorithms with the use case of optimization algorithms in mind. Based on this model, we present PAC-Bayesian generalization bounds for functions that are defined on the trajectory…
We consider periodic Markov chains with absorption. Applying to iterates of this periodic Markov chain criteria for the exponential convergence of conditional distributions of aperiodic absorbed Markov chains, we obtain exponential…
We study properties and parameter estimation of finite-state homogeneous continuous-time bivariate Markov chains. Only one of the two processes of the bivariate Markov chain is observable. The general form of the bivariate Markov chain…
In this paper, we propose a new policy iteration algorithm to compute the value function and the optimal controls of continuous time stochastic control problems. The algorithm relies on successive approximations using linear-quadratic…