Related papers: Variational Walkback: Learning a Transition Operat…

Graph models provide efficient tools to capture the underlying structure of data defined over networks. Many real-world network topologies are subject to change over time. Learning to model the dynamic interactions between entities in such…

We establish a general form of explicit, input-dependent, measure-valued warpings for learning nonstationary kernels. While stationary kernels are ubiquitous and simple to use, they struggle to adapt to functions that vary in smoothness…

The preparation of the stationary distribution of irreducible, time-reversible Markov chains is a fundamental building block in many heuristic approaches to algorithmically hard problems. It has been conjectured that quantum analogs of…

We introduce a unified operator-theoretic framework for analyzing mixing times of finite-state ergodic Markov chains that applies to both reversible and non-reversible dynamics. The central object in our analysis is the projected transition…

Recently, a class of stochastic processes known as piecewise deterministic Markov processes has been used to define continuous-time Markov chain Monte Carlo algorithms with a number of attractive properties, including compatibility with…

One of the proposed solutions to the equilibrium selection problem for agents learning in repeated games is obtained via the notion of stochastic stability. Learning algorithms are perturbed so that the Markov chain underlying the learning…

We study the problem of sequentially testing whether a given stochastic process is generated by a known Markov chain. Formally, given access to a stream of random variables, we want to quickly determine whether this sequence is a trajectory…

We develop a new bidirectional algorithm for estimating Markov chain multi-step transition probabilities: given a Markov chain, we want to estimate the probability of hitting a given target state in $\ell$ steps after starting from a given…

Markov chain Monte Carlo algorithms are invaluable tools for exploring stationary properties of physical systems, especially in situations where direct sampling is unfeasible. Common implementations of Monte Carlo algorithms employ…

In this paper, we present an overview of different types of random walk strategies with local and non-local transitions on undirected connected networks. We present a general approach to analyzing these strategies by defining the dynamics…

Queue networks describe complex stochastic systems of both theoretical and practical interest. They provide the means to assess alterations, diagnose poor performance and evaluate robustness across sets of interconnected resources. In the…

We present a new method, Non-Stationary Forward Flux Sampling, that allows efficient simulation of rare events in both stationary and non-stationary stochastic systems. The method uses stochastic branching and pruning to achieve uniform…

Offline Reinforcement learning is commonly used for sequential decision-making in domains such as healthcare and education, where the rewards are known and the transition dynamics $T$ must be estimated on the basis of batch data. A key…

There are multiple ways in which a stochastic system can be out of statistical equilibrium. It might be subject to time-varying forcing; or be in a transient phase on its way towards equilibrium; it might even be in equilibrium without us…

State-space models have been successfully used for more than fifty years in different areas of science and engineering. We present a procedure for efficient variational Bayesian learning of nonlinear state-space models based on sparse…

Generative flow networks (GFlowNets) are a method for learning a stochastic policy for generating compositional objects, such as graphs or strings, from a given unnormalized density by sequences of actions, where many possible action…

We develop a practical approach to establish the stability, that is, the recurrence in a given set, of a large class of controlled Markov chains. These processes arise in various areas of applied science and encompass important numerical…

The reconstruction and inference of stochastic dynamical systems from data is a fundamental task in inverse problems and statistical learning. While surrogate modeling advances computational methods to approximate these dynamics, standard…

We study the properties of a subclass of stochastic processes called discrete time nonlinear Markov chains with an aggregator, which naturally appear in various topics such as strategic queueing systems, inventory dynamics, opinion…

We study a special case of the problem of statistical learning without the i.i.d. assumption. Specifically, we suppose a learning method is presented with a sequence of data points, and required to make a prediction (e.g., a classification)…