Related papers: Supplement to "Markov Chain Monte Carlo Based on D…

In this paper, we give a new class of reconstructible graphs, which is an extension of my paper `A class of reconstructible graphs'.

This article presents a short and concise description of stochastic approximation algorithms in reinforcement learning of Markov decision processes. The algorithms can also be used as a suboptimal method for partially observed Markov…

In this paper, an approach to estimating a nonlinear deterministic model is presented. We introduce a stochastic model with extremely small variances so that the deterministic and stochastic models are essentially indistinguishable from…

We present a geometrically enhanced Markov chain Monte Carlo sampler for networks based on a discrete curvature measure defined on graphs. Specifically, we incorporate the concept of graph Forman curvature into sampling procedures on both…

To appear in the second edition of the MCMC handbook, S. P. Brooks, A. Gelman, G. Jones and X.-L. Meng (eds), Chapman & Hall.

The contents are divided into two papers "The Monotone Cumulants" (arXiv:0907.4896) and "Conditionally monotone independence" (arXiv:0907.5473).

We give computable bounds on the rate of convergence of the transition probabilities to the stationary distribution for a certain class of geometrically ergodic Markov chains. Our results are different from earlier estimates of Meyn and…

This erratum corrects the article arXiv:0806.2044 published in Ann. Probab. 36 (2008) 931--970

This article is a lecture note on the potential theory of (possibly non-reversible) Markov processes and on the connection of this theory with quantitative analysis of the metastability of stochastic processes.

In this paper we propose an efficient variance reduction approach for additive functionals of Markov chains relying on a novel discrete time martingale representation. Our approach is fully non-asymptotic and does not require the knowledge…

Discrete choice models are commonly used by applied statisticians in numerous fields, such as marketing, economics, finance, and operations research. When agents in discrete choice models are assumed to have differing preferences, exact…

Bayesian analysis often concerns an evaluation of models with different dimensionality as is necessary in, for example, model selection or mixture models. To facilitate this evaluation, transdimensional Markov chain Monte Carlo (MCMC)…

This paper focuses on time-varying delayed stochastic differential systems with stochastically switching parameters formulated by a unified switching behavior combining a discrete adapted process and a Cox process. Unlike prior studies…

This project is going to work with one example of stochastic matrix to understand how Markov chains evolve and how to use them to make faster and better decisions only looking to the present state of the system.

This paper specifies a notation for Markov decision processes.

This paper is intended to appear as a chapter for the Handbook of Markov Chain Monte Carlo. The goal of this chapter is to unify various problems at the intersection of Markov chain Monte Carlo (MCMC) and machine…

The purpose of this note is to correct an inaccuracy in the paper: R.P. Agaev and P.Yu. Chebotarev, "On Determining the Eigenprojection and Components of a Matrix," Autom. Remote Control, 2002, vol. 63, pp. 1537-1545 [arXiv:math/0508197],…

This version has been withdrawn. The new and final version is on ArXiv 1103.4878

An analytical approach to convolution of functions, which appear in perturbative calculations, is discussed. An extended list of integrals is presented.

The manuscript generalizes several recent results of the 2nd author concerning Markov-Chain Modelling of $(1,\lambda )$-ES Linear Optimization.