Related papers: Asymptotically Optimal Sequential Testing with Mar…
We consider the problem of sequentially testing a simple null hypothesis versus a composite alternative hypothesis that consists of a finite set of densities. We study sequential tests that are based on thresholding of mixture-based…
This paper investigates methods for estimating the optimal stochastic control policy for a Markov Decision Process with unknown transition dynamics and an unknown reward function. This form of model-free reinforcement learning comprises…
We study a correlated group testing model where items are infected according to a Markov chain, which creates bursty binfection patterns. Focusing on a very sparse infections regime, we propose a non adaptive testing strategy with an…
In pure exploration problems, a statistician sequentially collects information to answer a question about some stochastic and unknown environment. The probability of returning a wrong answer should not exceed a maximum risk parameter…
This article is concerned with stability analysis and stabilization of randomly switched nonlinear systems. These systems may be regarded as piecewise deterministic stochastic systems: the discrete switches are triggered by a stochastic…
For a reversible and ergodic Markov chain $\{X_n,n\geq0\}$ with invariant distribution $\pi$, we show that a valid confidence interval for $\pi(h)$ can be constructed whenever the asymptotic variance $\sigma^2_P(h)$ is finite and positive.…
We propose a novel flexible-step model predictive control algorithm for unknown linear time-invariant discrete-time systems. The goal is to asymptotically stabilize the system without relying on a pre-collected dataset that describes its…
Hybrid systems whose mode dynamics are governed by non-linear ordinary differential equations (ODEs) are often a natural model for biological processes. However such models are difficult to analyze. To address this, we develop a…
Ergodic properties and asymptotic stationarity are investigated in this paper for the pseudo-covariance matrix (PCM) of a recursive state estimator which is robust against parametric uncertainties and is based on plant output measurements…
We present some new results on sample path optimality for the ergodic control problem of a class of non-degenerate diffusions controlled through the drift. The hypothesis most often used in the literature to ensure the existence of an a.s.…
We develop a class of optimal tests for a structural break occurring at an unknown date in infinite and growing-order time series regression models, such as AR($\infty$), linear regression with increasingly many covariates, and…
Estimating the transition dynamics of controlled Markov chains is crucial in fields such as time series analysis, reinforcement learning, and system exploration. Traditional non-parametric density estimation methods often assume independent…
We consider the parameter estimation of Markov chain when the unknown transition matrix belongs to an exponential family of transition matrices. Then, we show that the sample mean of the generator of the exponential family is an…
Given an observation $\mathbf Y \in \mathbb{R}^{d_1\times d_2}$ from the model $\mathbf Y = \mathbf X + \mathbf E$ where $\mathbf X$ is constant and $\mathbf E$ has i.i.d. $N(0,1)$ entries, we consider the problem of detecting a planted…
Suppose that at any stage of a statistical experiment a control variable $X$ that affects the distribution of the observed data $Y$ can be used. The distribution of $Y$ depends on some unknown parameter $\theta$, and we consider the…
When the initial and transition probabilities of a finite Markov chain in discrete time are not well known, we should perform a sensitivity analysis. This can be done by considering as basic uncertainty models the so-called credal sets that…
We revisit the source coding problem for a Markov chain under the assumption that the transmission times and how fast the Markov chain transitions its state happen at the same time-scale. Specifically, we assume that the transmission of…
We consider continuous-time Markov chain on a finite state space X. We assume X can be clustered into several subsets such that the intra-transition rates within these subsets are of order $\mathcal{O}(\frac{1}{\epsilon})$ comparing to the…
We present an alternative view for the study of optimal control of partially observed Markov Decision Processes (POMDPs). We first revisit the traditional (and by now standard) separated-design method of reducing the problem to fully…
In this paper, we consider semi-Markov processes whose transition times and transition probabilities depend on a small parameter $\varepsilon$. Understanding the asymptotic behavior of such processes is needed in order to study the…