Related papers: Learning Multiple Markov Chains via Adaptive Alloc…
Time-homogeneous Markov chains are often used as disease progression models in studies of cost-effectiveness and optimal decision-making. Maximum likelihood estimation of these models can be challenging when data are collected at a time…
We study the problem of sequentially testing whether a given stochastic process is generated by a known Markov chain. Formally, given access to a stream of random variables, we want to quickly determine whether this sequence is a trajectory…
Adaptive Markov chains are an important class of Monte Carlo methods for sampling from probability distributions. The time evolution of adaptive algorithms depends on past samples, and thus these algorithms are non-Markovian. Although there…
This work includes a number of novel contributions for the multiple-source adaptation problem. We present new normalized solutions with strong theoretical guarantees for the cross-entropy loss and other similar losses. We also provide new…
We study the multi-task learning problem that aims to simultaneously analyze multiple datasets collected from different sources and learn one model for each of them. We propose a family of adaptive methods that automatically utilize…
Simultaneous load forecasting across multiple entities (e.g., regions, buildings) is crucial for the efficient, reliable, and cost-effective operation of power systems. Accurate load forecasting is a challenging problem due to the inherent…
Most algorithms for decentralized learning employ a consensus or diffusion mechanism to drive agents to a common solution of a global optimization problem. Generally this takes the form of linear averaging, at a rate of contraction…
One of the major problems in adaptive filtering is the problem of system identification. It has been studied extensively due to its immense practical importance in a variety of fields. The underlying goal is to identify the impulse response…
The conventional perspective on Markov chains considers decision problems concerning the probabilities of temporal properties being satisfied by traces of visited states. However, consider the following query made of a stochastic system…
We consider the problem of defining and fitting models of autoregressive time series of probability distributions on a compact interval of $\mathbb{R}$. An order-$1$ autoregressive model in this context is to be understood as a Markov…
We develop and analyze a general technique for learning with an unknown distribution drift. Given a sequence of independent observations from the last $T$ steps of a drifting distribution, our algorithm agnostically learns a family of…
In this paper we develop a statistical estimation technique to recover the transition kernel $P$ of a Markov chain $X=(X_m)_{m \in \mathbb N}$ in presence of censored data. We consider the situation where only a sub-sequence of $X$ is…
We consider the problem of finding the transition rates of a continuous-time homogeneous Markov chain under the empirical condition that the state changes at most once during a time interval of unit length. It is proven that this…
The basic question in perturbation analysis of Markov chains is: how do small changes in the transition kernels of Markov chains translate to chains in their stationary distributions? Many papers on the subject have shown, roughly, that the…
We study very simple sorting algorithms based on a probabilistic comparator model. In our model, errors in comparing two elements are due to (1) the energy or effort put in the comparison and (2) the difference between the compared…
In this paper, we propose an approach to obtain reduced-order models of Markov chains. Our approach is composed of two information-theoretic processes. The first is a means of comparing pairs of stationary chains on different state spaces,…
Reinforcement learning agents have been mostly developed and evaluated under the assumption that they will operate in a fully autonomous manner -- they will take all actions. In this work, our goal is to develop algorithms that, by learning…
We consider the problem of estimating the expected time to find a maximum degree node on a graph using a (parameterized) biased random walk. For assortative graphs the positive degree correlation serves as a local gradient for which a bias…
Consider a system evolving according to an absorbing discrete-time Markov chain with known transition matrix. The state of the system is observed at two points in time, separated by an unknown number of generations. We are interested in…
We consider the problem of approximating the stationary distribution of an ergodic Markov chain given a set of sampled transitions. Classical simulation-based approaches assume access to the underlying process so that trajectories of…