相关论文: One-sided Markov additive processes with lattice a…

Markov branching systems form a fundamental class of stochastic models that are extensively applied in biology, physics, finance, and other domains. These systems are distinguished by their continuous-time evolution and inherent branching…

Markov Decision Processes (MDPs) have been used to formulate many decision-making problems in science and engineering. The objective is to synthesize the best decision (action selection) policies to maximize expected rewards (or minimize…

Expanding upon the rich history of algebraic techniques in probability, we show the existence of and construct a Markov chain using the Hopf square map on a quantum group that is both non-commutative and non-cocommutative. This extends the…

In this paper, we outline a model of graph (or network) dynamics based on two ingredients. The first ingredient is a Markov chain on the space of possible graphs. The second ingredient is a semi-Markov counting process of renewal type. The…

Markov jump processes are continuous-time stochastic processes which describe dynamical systems evolving in discrete state spaces. These processes find wide application in the natural sciences and machine learning, but their inference is…

We show that nonequilibrium dynamics can play a constructive role in unsupervised machine learning by inducing the spontaneous emergence of latent-state cycles. We introduce a model in which visible and hidden variables interact through two…

Markov chains are fundamental models for stochastic dynamics, with applications in a wide range of areas such as population dynamics, queueing systems, reinforcement learning, and Monte Carlo methods. Estimating the transition matrix and…

Markov chain Monte Carlo is an inherently serial algorithm. Although likelihood calculations for individual steps can sometimes be parallelized, the serial evolution of the process is widely viewed as incompatible with parallelization,…

Given noisy, partial observations of a time-homogeneous, finite-statespace Markov chain, conceptually simple, direct statistical inference is available, in theory, via its rate matrix, or infinitesimal generator, $\mathsf{Q}$, since $\exp…

This paper provides full classification of dynamics for continuous time Markov chains (CTMCs) on the non-negative integers with polynomial transition rate functions. Such stochastic processes are abundant in applications, in particular in…

We propose an interacting many-body space-time-discrete Markov chain model, which is composed of an integrable deterministic and reversible cellular automaton (the rule 54 of [Bobenko et al, CMP 158, 127 (1993)]) on a finite one-dimensional…

Consider a Markov chain $\{X_n\}_{n\ge 0}$ with an ergodic probability measure $\pi$. Let $\Psi$ a function on the state space of the chain, with $\alpha$-tails with respect to $\pi$, $\alpha\in (0,2)$. We find sufficient conditions on the…

Markov chain analysis is a key technique in formal verification. A practical obstacle is that all probabilities in Markov models need to be known. However, system quantities such as failure rates or packet loss ratios, etc. are often not --…

We use an extension of fundamental measure theory to lattice hard-core fluids to study the phase diagram of two different systems. First, two-dimensional parallel hard squares with edge-length $\sigma=2$ in a simple square lattice. This…

We propose a new approach for estimating the finite dimensional transition matrix of a Markov chain using a large number of independent sample paths observed at random times. The sample paths may be observed as few as two times, and the…

We extend the Markov additive methodology developed in [Ann. Appl. Probab. 9 (1999) 110-145, Ann. Appl. Probab. 11 (2001) 596-607] to obtain the sharp asymptotics of the steady state probability of a queueing network when one of the nodes…

Markov chains are a natural and well understood tool for describing one-dimensional patterns in time or space. We show how to infer $k$-th order Markov chains, for arbitrary $k$, from finite data by applying Bayesian methods to both…

We analyze asymptotically a differential-difference equation, that arises in a Markov-modulated fluid model. We use singular perturbation methods to analyze the problem with appropriate scalings of the two state variables. In particular,…

Lattice simulations employing reweighting and Taylor expansion techniques have predicted a (\mu,T)-phase diagram according to general expectations, with an analytic quark-hadron crossover at \mu=0 turning into a first order transition at…

A rescaled Markov chain converges uniformly in probability to the solution of an ordinary differential equation, under carefully specified assumptions. The presentation is much simpler than those in the outside literature. The result may be…