Related papers: The Arnoldi Aggregation for Approximate Transient …
We investigate absorption, i.e., almost sure convergence to an absorbing state, in time-varying (non-homogeneous) discrete-time Markov chains with finite state space. We consider systems that can switch among a finite set of transition…
Sampling from the stationary distribution is one of the fundamental tasks of Markov chain-based algorithms and has important applications in machine learning, combinatorial optimization and network science. For the quantum case, qsampling…
In this paper we show how to generalize the quantum approximate counting technique developed by Brassard, H{\o}yer and Tapp [ICALP 1998] to a more general setting: estimating the number of marked states of a Markov chain (a Markov chain can…
This paper calculates transient distributions of a special class of Markov processes with continuous state space and in continuous time, up to an explicit error bound. We approximate specific queues on R with one-sided L\'evy input, such as…
This paper is concerned with the development of rigorous approximations to various expectations associated with Markov chains and processes having non-stationary transition probabilities. Such non-stationary models arise naturally in…
We develop exact simulation (also known as perfect sampling) algorithms for a family of assemble-to-order systems. Due to the finite capacity, and coupling in demands and replenishments, known solving techniques are inefficient for larger…
In this paper we propose an efficient variance reduction approach for additive functionals of Markov chains relying on a novel discrete time martingale representation. Our approach is fully non-asymptotic and does not require the knowledge…
Operator-theoretic analysis of nonlinear dynamical systems has attracted much attention in a variety of engineering and scientific fields, endowed with practical estimation methods using data such as dynamic mode decomposition. In this…
We investigate the problem of quantifying contraction coefficients of Markov transition kernels in Kantorovich ($L^1$ Wasserstein) distances. For diffusion processes, relatively precise quantitative bounds on contraction rates have recently…
We study the properties of a subclass of stochastic processes called discrete time nonlinear Markov chains with an aggregator, which naturally appear in various topics such as strategic queueing systems, inventory dynamics, opinion…
Value iteration is a well-known method of solving Markov Decision Processes (MDPs) that is simple to implement and boasts strong theoretical convergence guarantees. However, the computational cost of value iteration quickly becomes…
In this paper we consider the problem of graph-based transductive classification, and we are particularly interested in the directed graph scenario which is a natural form for many real world applications. Different from existing research…
This paper surveys various results about Markov chains on general (non-countable) state spaces. It begins with an introduction to Markov chain Monte Carlo (MCMC) algorithms, which provide the motivation and context for the theory which…
Consider a finite irreducible Markov chain with invariant distribution $\pi$. We use the inner product induced by $\pi$ and the associated heat operator to simplify and generalize some results related to graph partitioning and the small-set…
Continuous-time Markov chains describing interacting processes exhibit a state space that grows exponentially in the number of processes. This state-space explosion renders the computation or storage of the time-marginal distribution, which…
We consider the consensual distributed optimization problem and propose an asynchronous version of the Alternating Direction Method of Multipliers (ADMM) algorithm to solve it. The `asynchronous' part here refers to the fact that only one…
A sketch-and-select Arnoldi process to generate a well-conditioned basis of a Krylov space at low cost is proposed. At each iteration the procedure utilizes randomized sketching to select a limited number of previously computed basis…
A mapping of the process on a continuous configuration space to the symbolic representation of the motion on a discrete state space will be combined with an iterative aggregation and disaggregation (IAD) procedure to obtain steady state…
We propose a method to sample stationary properties of solutions of stochastic differential equations, which is accurate and efficient if there are rarely visited regions or rare transitions between distinct regions of the state space. The…
Branching processes are a class of continuous-time Markov chains (CTMCs) prevalent for modeling stochastic population dynamics in ecology, biology, epidemiology, and many other fields. The transient or finite-time behavior of these systems…