Related papers: Separating effect from significance in Markov chai…
A general theory is developed to study individual based models which are discrete in time. We begin by constructing a Markov chain model that converges to a one-dimensional map in the infinite population limit. Stochastic fluctuations are…
A method of constructing Markov chains on finite state spaces is provided. The chain is specified by three constraints: stationarity, dependence and marginal distributions. The generalized Pythagorean theorem in information geometry plays a…
A decade ago, Abdulla, Ben Henda and Mayr introduced the elegant concept of decisiveness for denumerable Markov chains [1]. Roughly speaking, decisiveness allows one to lift most good properties from finite Markov chains to denumerable…
A comparison theorem for state-dependent regime-switching diffusion processes is established, which enables us to control pathwisely the evolution of the state-dependent switching component simply by Markov chains. Moreover, a sharp…
We study the problem of identity testing of markov chains. In this setting, we are given access to a single trajectory from a markov chain with unknown transition matrix $Q$ and the goal is to determine whether $Q = P$ for some known matrix…
In recent decades, a number of profound theorems concerning approximation of hard counting problems have appeared. These include estimation of the permanent, estimating the volume of a convex polyhedron, and counting (approximately) the…
In this work we introduce new approximate similarity relations that are shown to be key for policy (or control) synthesis over general Markov decision processes. The models of interest are discrete-time Markov decision processes, endowed…
A Peskun ordering between two samplers, implying a dominance of one over the other, is known among the Markov chain Monte Carlo community for being a remarkably strong result. It is however also known for being a result that is notably…
From the perspective of probability, the stability of growing network is studied in the present paper. Using the DMS model as an example, we establish a relation between the growing network and Markov process. Based on the concept and…
Policy gradient methods are among the most effective methods in challenging reinforcement learning problems with large state and/or action spaces. However, little is known about even their most basic theoretical convergence properties,…
We consider the problem of statistical inference in a parametric finite Markov chain model and develop a robust estimator of the parameters defining the transition probabilities via minimization of a suitable (empirical) version of the…
We introduce an statistical mechanical formalism for the study of discrete-time stochastic processes with which we prove: (i) General properties of extremal chains, including triviality on the tail $\sigma$-algebra, short-range…
We develop a model for credit rating migration that accounts for the impact of economic state fluctuations on default probabilities. The joint process for the economic state and the rating is modelled as a time-homogeneous Markov chain.…
Filtering is concerned with the sequential estimation of the state, and uncertainties, of a Markovian system, given noisy observations. It is particularly difficult to achieve accurate filtering in complex dynamical systems, such as those…
A statistical test is presented to decide whether data are adequately described by probabilistic functions of finite state Markov chains (''hidden Markov models'') as applied in the analysis of ion channel data. Particularly, the test can…
We revisit the classical problem of approximating a stochastic differential equation by a discrete-time and discrete-space Markov chain. Our construction iterates Caratheodory's theorem over time to match the moments of the increments…
Gerrymandering is a practice of manipulating district boundaries and locations in order to achieve a political advantage for a particular party. Lewenberg, Lev, and Rosenschein [AAMAS 2017] initiated the algorithmic study of a…
A new object of the probability theory, two-sided chain of events (symbols), is introduced. A theory of multi-steps Markov chains with long-range memory, proposed earlier in Phys. Rev. E 68, 06117 (2003), is developed and used to establish…
We prove deviation bounds for the random variable $\sum_{i=1}^{n} f_i(Y_i)$ in which $\{Y_i\}_{i=1}^{\infty}$ is a Markov chain with stationary distribution and state space $[N]$, and $f_i: [N] \rightarrow [-a_i, a_i]$. Our bound improves…
We discuss the efficient computation of performance, reliability, and availability measures for Markov chains; these metrics, and the ones obtained by combining them, are often called performability measures. We show that this computational…