Related papers: Asymptotically Optimal Sequential Testing with Mar…
We study sequential multiple testing with independent data streams, where the goal is to identify an unknown subset of signals while controlling commonly used error metrics, including generalized familywise rates and false discovery and…
We exhibit an efficient procedure for testing, based on a single long state sequence, whether an unknown Markov chain is identical to or $\varepsilon$-far from a given reference chain. We obtain nearly matching (up to logarithmic factors)…
New goodness-of-fit tests for Markovian models in time series analysis are developed which are based on the difference between a fully nonparametric estimate of the one-step transition distribution function of the observed process and that…
A multi-armed bandit with finitely many arms is studied when each arm is a homogeneous Markov process on an underlying finite state space. The transition law of one of the arms, referred to as the odd arm, is different from the common…
We consider stochastic optimization problems where data is drawn from a Markov chain. Existing methods for this setting crucially rely on knowing the mixing time of the chain, which in real-world applications is usually unknown. We propose…
Time estimation is a fundamental task that underpins precision measurement, global navigation systems, financial markets, and the organisation of everyday life. Many biological processes also depend on time estimation by nanoscale clocks,…
This article develops a method to construct the optimal sequential test for monitoring the changes in the distribution of finite observation sequences with a general dependence structure. This method allows us to prove that different…
We introduce the concept of asymptotic period for an irreducible and aperiodic, discrete-time Markov chain X on a countable state space, and develop the theory leading to its formal definition. The asymptotic period of X equals one - its…
Observing stochastic trajectories with rare transitions between states, practically undetectable on time scales accessible to experiments, makes it impossible to directly quantify the entropy production and thus infer whether and how far…
Markov chain Monte Carlo(MCMC) is a popular approach to sample from high dimensional distributions, and the asymptotic variance is a commonly used criterion to evaluate the performance. While most popular MCMC algorithms are reversible,…
We present a novel algorithm to solve a non-linear system of equations, whose solution can be interpreted as a tight lower bound on the vector of expected hitting times of a Markov chain whose transition probabilities are only partially…
We study the problem of learning the transition matrices of a set of Markov chains from a single stream of observations on each chain. We assume that the Markov chains are ergodic but otherwise unknown. The learner can sample Markov chains…
Optimal designs minimize the number of experimental runs (samples) needed to accurately estimate model parameters, resulting in algorithms that, for instance, efficiently minimize parameter estimate variance. Governed by knowledge of past…
This article provides the first procedure for computing a fully data-dependent interval that traps the mixing time $t_{\text{mix}}$ of a finite reversible ergodic Markov chain at a prescribed confidence level. The interval is computed from…
A random sequence having two segments being the homogeneous Markov processes is registered. Each segment has his own transition probability law and the length of the segment is unknown and random. The transition probabilities of each…
We consider the problem of estimating the asymptotic variance of a function defined on a Markov chain, an important step for statistical inference of the stationary mean. We design a novel recursive estimator that requires $O(1)$…
We investigate the problem of jointly testing multiple hypotheses and estimating a random parameter of the underlying distribution in a sequential setup. The aim is to jointly infer the true hypothesis and the true parameter while using on…
A novel sequential change detection problem is proposed, in which the goal is to not only detect but also accelerate the change. Specifically, it is assumed that the sequentially collected observations are responses to treatments selected…
A novel data-driven method for formal verification is proposed to study complex systems operating in safety-critical domains. The proposed approach is able to formally verify discrete-time stochastic dynamical systems against temporal logic…
Markov chains can be used to generate samples whose distribution approximates a given target distribution. The quality of the samples of such Markov chains can be measured by the discrepancy between the empirical distribution of the samples…