Related papers: Additive N-Step Markov Chains as Prototype Model o…
This article presents several results establishing connections be- tween Markov chains and dynamical systems, from the point of view of open systems in physics. We show how all Markov chains can be understood as the information on one…
Stochastic processes find applications in modelling systems in a variety of disciplines. A large number of stochastic models considered are Markovian in nature. It is often observed that higher order Markov processes can model the data…
The goal of this paper is to analyze distributional Markov Decision Processes as a class of control problems in which the objective is to learn policies that steer the distribution of a cumulative reward toward a prescribed target law,…
We address the problem of long-range memory in the financial markets. There are two conceptually different ways to reproduce power-law decay of auto-correlation function: using fractional Brownian motion as well as non-linear stochastic…
We provide a comprehensive overview of latent Markov (LM) models for the analysis of longitudinal categorical data. The main assumption behind these models is that the response variables are conditionally independent given a latent process…
In this paper we propose a model for open Markov chains that can be interpreted as a system of non-interacting particles evolving according to the rules of a Markov chain. The number of particles in the system is not constant, because we…
One of the proposed solutions to the equilibrium selection problem for agents learning in repeated games is obtained via the notion of stochastic stability. Learning algorithms are perturbed so that the Markov chain underlying the learning…
A piecewise-deterministic Markov process, specified by random jumps and switching semi-flows, as well as the associated Markov chain given by its post-jump locations, are investigated in this paper. The existence of an exponentially…
We propose a novel framework for the Lyapunov analysis of an important class of hybrid systems, inspired by the theory of symbolic dynamics and earlier results on the restricted class of switched systems. This new framework allows us to…
The trajectories of diffusion processes are continuous but non-differentiable, and each occurs with vanishing probability. This introduces a gap between theory, where path probabilities are used in many contexts, and experiment, where only…
An exact description of the statistical motion of active particles in three dimension is presented in the framework of a generalized diffusion equation. Such a generalization contemplates a non-local, in time and space, connecting (memory)…
We develop a systematic approach to the linear-noise approximation for stochastic reaction systems with distributed delays. Unlike most existing work our formalism does not rely on a master equation, instead it is based upon a dynamical…
A sequence of real numbers (x_n) is Benford if the significands, i.e. the fraction parts in the floating-point representation of (x_n) are distributed logarithmically. Similarly, a discrete-time irreducible and aperiodic finite-state Markov…
We introduce neural Markov logic networks (NMLNs), a statistical relational learning system that borrows ideas from Markov logic. Like Markov logic networks (MLNs), NMLNs are an exponential-family model for modelling distributions over…
The paper studies an improved estimate for the rate of convergence for nonlinear homogeneous discrete-time Markov chains. These processes are nonlinear in terms of the distribution law. Hence, the transition kernels are dependent on the…
We study long time behavior of a discrete time weakly interacting particle system, and the corresponding nonlinear Markov process in $\mathbb{R}^d$, described in terms of a general stochastic evolution equation. In a setting where the state…
Neural network language models (NNLMs) have achieved ever-improving accuracy due to more sophisticated architectures and increasing amounts of training data. However, the inductive bias of these models (formed by the distributional…
We develop a formalism to describe the discrete-time dynamics of systems containing an arbitrary number of interacting species. The individual-based model, which forms our starting point, is described by a Markov chain, which in the limit…
For a series of Markov processes we prove stochastic duality relations with duality functions given by orthogonal polynomials. This means that expectations with respect to the original process (which evolves the variable of the orthogonal…
We consider a class of stochastic dynamical systems, called piecewise deterministic Markov processes, with states $(x, \s)\in \O\times \G$, $\O$ being a region in $\bbR^d$ or the $d$--dimensional torus, $\G$ being a finite set. The…