Related papers: Differential equation approximations for Markov ch…

This project is going to work with one example of stochastic matrix to understand how Markov chains evolve and how to use them to make faster and better decisions only looking to the present state of the system.

Ordinary differential equations obtained as limits of Markov processes appear in many settings. They may arise by scaling large systems, or by averaging rapidly fluctuating systems, or in systems involving multiple time-scales, by a…

We present new algorithms and fast implementations to find efficient approximations for modelling stochastic processes. For many numerical computations it is essential to develop finite approximations for stochastic processes. While the…

In this note, a general approach to the study of non-stationary Markov chains with catastrophes and the corresponding queuing models is considered, as well as to obtain estimates of the limiting regime itself. As an illustration, an example…

The Markov chain approximation of a one-dimensional symmetric diffusion is investigated in this paper. Given an irreducible reflecting diffusion on a closed interval with scale function $s$ and speed measure $m$, the approximating Markov…

Computational procedures for the stationary probability distribution, the group inverse of the Markovian kernel and the mean first passage times of an irreducible Markov chain, are developed using perturbations. The derivation of these…

We propose a new approach for estimating the finite dimensional transition matrix of a Markov chain using a large number of independent sample paths observed at random times. The sample paths may be observed as few as two times, and the…

The two-parameter Macdonald polynomials are a central object of algebraic combinatorics and representation theory. We give a Markov chain on partitions of k with eigenfunctions the coefficients of the Macdonald polynomials when expanded in…

Weak approximations have been developed to calculate the expectation value of functionals of stochastic differential equations, and various numerical discretization schemes (Euler, Milshtein) have been studied by many authors. We present a…

The theory of stochastic approximations form the theoretical foundation for studying convergence properties of many popular recursive learning algorithms in statistics, machine learning and statistical physics. Large deviations for…

Markov chain models are used in various fields, such behavioral sciences or econometrics. Although the goodness of fit of the model is usually assessed by large sample approximation, it is desirable to use conditional tests if the sample…

Computing the stationary distributions of a continuous-time Markov chain (CTMC) involves solving a set of linear equations. In most cases of interest, the number of equations is infinite or too large, and the equations cannot be solved…

Many random processes can be simulated as the output of a deterministic model accepting random inputs. Such a model usually describes a complex mathematical or physical stochastic system and the randomness is introduced in the input…

Global variational approximation methods in graphical models allow efficient approximate inference of complex posterior distributions by using a simpler model. The choice of the approximating model determines a tradeoff between the…

In the present paper, we give some examples of stochastic differential equations which have delicateness in the Markov and strong Markov properties, the uniqueness locally in time and globally in time, and initial conditions. Moreover, we…

It has become increasingly easy nowadays to collect approximate posterior samples via fast algorithms such as variational Bayes, but concerns exist about the estimation accuracy. It is tempting to build solutions that exploit approximate…

We consider a Markov control model in discrete time with countable both state space and action space. Using the value function of a suitable long-run average reward problem, we study various reachability/controllability problems. First, we…

For Markov chains and Markov processes exhibiting a form of stochastic monotonicity (larger states shift up transition probabilities in terms of stochastic dominance), stability and ergodicity results can be obtained using order-theoretic…

We present an algorithm that can efficiently compute a broad class of inferences for discrete-time imprecise Markov chains, a generalised type of Markov chains that allows one to take into account partially specified probabilities and other…

Markov chains are a class of probabilistic models that have achieved widespread application in the quantitative sciences. This is in part due to their versatility, but is compounded by the ease with which they can be probed analytically.…