Related papers: The parallel replica method for simulating long tr…

We propose a computational method for large deviation statistics of time-averaged quantities in general Markov processes. In our proposed method, we repeat a response measurement against external forces, where the forces are determined by…

Markov chain Monte Carlo (MCMC), such as Langevin dynamics, is valid for approximating intractable distributions. However, its usage is limited in the context of deep latent variable models owing to costly datapoint-wise sampling iterations…

We study computational and statistical aspects of learning Latent Markov Decision Processes (LMDPs). In this model, the learner interacts with an MDP drawn at the beginning of each epoch from an unknown mixture of MDPs. To sidestep known…

Auxiliary variable methods such as the Parallel Tempering and the cluster Monte Carlo methods generate samples that follow a target distribution by using proposal and auxiliary distributions. In sampling from complex distributions, these…

We consider multiple parallel Markov decision processes (MDPs) coupled by global constraints, where the time varying objective and constraint functions can only be observed after the decision is made. Special attention is given to how well…

Importance sampling of trajectories has proved a uniquely successful strategy for exploring rare dynamical behaviors of complex systems in an unbiased way. Carrying out this sampling, however, requires an ability to propose changes to…

Leveraging recent work on data-driven methods for constructing a finite state space Markov process from dynamical systems, we address two problems for obtaining further reduced statistical representations. The first problem is to extract…

This paper proposes a general formulation for temporal parallelisation of dynamic programming for optimal control problems. We derive the elements and associative operators to be able to use parallel scans to solve these problems with…

Parallel algorithms designed for simulation and performance evaluation of single-server tandem queueing systems with both infinite and finite buffers are presented. The algorithms exploit a simple computational procedure based on recursive…

We consider parallel asynchronous Markov Chain Monte Carlo (MCMC) sampling for problems where we can leverage (stochastic) gradients to define continuous dynamics which explore the target distribution. We outline a solution strategy for…

Probabilistic models are conceptually powerful tools for finding structure in data, but their practical effectiveness is often limited by our ability to perform inference in them. Exact inference is frequently intractable, so approximate…

We consider large-scale Markov decision processes (MDPs) with a risk measure of variability in cost, under the risk-aware MDPs paradigm. Previous studies showed that risk-aware MDPs, based on a minimax approach to handling risk, can be…

Semi-Markov processes are Markovian processes in which the firing time of the transitions is modelled by probabilistic distributions over positive reals interpreted as the probability of firing a transition at a certain moment in time. In…

Replication of experimental results has been a challenge faced by many scientific disciplines, including the field of machine learning. Recent work on the theory of machine learning has formalized replicability as the demand that an…

Efficient sampling of many-dimensional and multimodal density functions is a task of great interest in many research fields. We describe an algorithm that allows parallelizing inherently serial Markov chain Monte Carlo (MCMC) sampling by…

This paper provides a general and abstract approach to approximate ergodic regimes of Markov and Feller processes. More precisely, we show that the recursive algorithm presented in Lamberton & Pages (2002) and based on simulation algorithms…

As it has become common to use many computer cores in routine applications, finding good ways to parallelize popular algorithms has become increasingly important. In this paper, we present a parallelization scheme for Markov chain Monte…

We propose Markov two-components processes (M2CP) as a probabilistic model of asynchronous systems based on the trace semantics for concurrency. Considering an asynchronous system distributed over two sites, we introduce concepts and tools…

In this paper we consider the problem of computing the stationary distribution of nearly completely decomposable Markov processes, a well-established area in the classical theory of Markov processes with broad applications in the design,…

We discuss a Monte Carlo Markov Chain (MCMC) procedure for the random sampling of some one-dimensional lattice paths with constraints, for various constraints. We show that an approach inspired by optimal transport allows us to bound…