Related papers: Nearest neighbor Markov dynamics on Macdonald proc…
This paper contributes to the emerging viewpoint that governing equations for dynamic state estimation, conditioned on the history of noisy measurements, can be viewed as gradient flow on the manifold of joint probability density functions…
This paper presents a general approach to linear stochastic processes driven by various random noises. Mathematically, such processes are described by linear stochastic differential equations of arbitrary order (the simplest non-trivial…
We consider Markov processes of cubic stochastic (in a fixed sense) matrices which are also called quadratic stochastic process (QSPs). A QSP is a particular case of a continuous-time dynamical system whose states are stochastic cubic…
In this paper, we derive a simple drift condition for the stability of a class of two-dimensional Markov processes, for which one of the coordinates (also referred to as the {\em phase} for convenience) has a well understood behaviour…
We consider Markov decision processes (MDP) as generators of sequences of probability distributions over states. A probability distribution is p-synchronizing if the probability mass is at least p in a single state, or in a given set of…
We identify the linear space spanned by the real-valued excessive functions of a Markov process with the set of those functions which are quasimartingales when we compose them with the process. Applications to semi-Dirichlet forms are…
In this work we introduce new approximate similarity relations that are shown to be key for policy (or control) synthesis over general Markov decision processes. The models of interest are discrete-time Markov decision processes, endowed…
We consider synthesis of control policies that maximize the probability of satisfying given temporal logic specifications in unknown, stochastic environments. We model the interaction between the system and its environment as a Markov…
We study upper and lower bounds on the sample-complexity of learning near-optimal behaviour in finite-state discounted Markov Decision Processes (MDPs). For the upper bound we make the assumption that each action leads to at most two…
The purpose of this work is to build a framework that allows for an in-depth study of various generalisations to inhomogeneous space of models of Borodin-Ferrari, Dieker-Warren, Nordenstam, Warren-Windridge of interacting particles in…
Hybrid systems whose mode dynamics are governed by non-linear ordinary differential equations (ODEs) are often a natural model for biological processes. However such models are difficult to analyze. To address this, we develop a…
We study the long-time behaviour of a class of piecewise-deterministic Markov processes which are an extension of some recent works. These $d$-dimensional processes, d>=1, can especially be used to model the motion of a bacterium in…
This paper generalizes the notion of stochastic order to a relation between probability measures over arbitrary measurable spaces. This generalization is motivated by the observation that for the stochastic ordering of two stationary Markov…
Spectral statistics of quantum systems have been studied in detail using the nearest neighbour level spacings, which for generic chaotic systems follows random matrix theory predictions. In this work, the probability density of the closest…
Interacting-Particle Reaction Dynamics (iPRD) simulates the spatiotemporal evolution of particles that experience interaction forces and can react with one another. The combination of interaction forces and reactions enable a wide range of…
The probability distributions, as well as the mean values of stochastic currents and fluxes, associated with a driven Langevin process, provide a good and topologically protected measure of how far a stochastic system is driven out of…
Entropy production characterizes irreversibility. This viewpoint allows us to consider the thermodynamic uncertainty relation, which states that a higher precision can be achieved at the cost of higher entropy production, as a relation…
One new thing often leads to another. Such correlated novelties are a familiar part of daily life. They are also thought to be fundamental to the evolution of biological systems, human society, and technology. By opening new possibilities,…
Markov decision processes (MDPs) with rewards are a widespread and well-studied model for systems that make both probabilistic and nondeterministic choices. A fundamental result about MDPs is that their minimal and maximal expected rewards…
We derive explicit upper bounds for the $\bar{d}$-distance between a chain of infinite order and its canonical $k$-steps Markov approximation. Our proof is entirely constructive and involves a "coupling from the past" argument. The new…