Related papers: When are Kalman-filter restless bandits indexable?
In this paper we analyze the convergence of a family of robust Kalman filters. For each filter of this family the model uncertainty is tuned according to the so called tolerance parameter. Assuming that the corresponding state-space model…
Restless multi-armed bandits with partially observable states has applications in communication systems, age of information and recommendation systems. In this paper, we study multi-state partially observable restless bandit models. We…
We consider Kalman filtering problems when the observations are intermittently erased or lost. It was known that the estimates are mean-square unstable when the erasure probability is larger than a certain critical value, and stable…
Restless bandit problems are instances of non-stationary multi-armed bandits. These problems have been studied well from the optimization perspective, where the goal is to efficiently find a near-optimal policy when system parameters are…
We demonstrate that the extended Kalman filter converges locally for a broad class of nonlinear systems. If the initial estimation error of the filter is not too large then the error goes to zero exponentially as time goes to infinity. To…
We consider a version of the continuous-time multi-armed bandit problem where decision opportunities arrive at Poisson arrival times, and study its Gittins index policy. When driven by spectrally one-sided L\'evy processes, the Gittins…
We consider a robust state space filtering problem in the case that the transition probability density is unknown and possibly degenerate. The resulting robust filter has a Kalman-like structure and solves a minimax game: the nature selects…
This article develops a comprehensive framework for stability analysis of a broad class of commonly used continuous and discrete time-filters for stochastic dynamic systems with non-linear state dynamics and linear measurements under…
This paper develops a connection between the asymptotic stability of nonlinear filters and a notion of observability. We consider a general class of hidden Markov models in continuous time with compact signal state space, and call such a…
The paper investigates recoverability of discrete time signals represented by infinite sequences from incomplte observations. It is shown that there exist wide classes of signals that are everywhere dense in the space of square-summable…
We consider a robust filtering problem where the robust filter is designed according to the least favorable model belonging to a ball about the nominal model. In this approach, the ball radius specifies the modeling error tolerance and the…
Restless bandits are an important class of problems with applications in recommender systems, active learning, revenue management and other areas. We consider infinite-horizon discounted restless bandits with many arms where a fixed…
This paper considers the Linear Minimum Variance recursive state estimation for the linear discrete time dynamic system with random state transition and measurement matrices, i.e., random parameter matrices Kalman filtering. It is shown…
This paper uses a probabilistic approach to analyze the converge of an ensemble Kalman filter solution to an exact Kalman filter solution in the simplest possible setting, the scalar case, as it allows us to build upon a rich literature of…
The nonlinear filter associated with the discrete time signal-observation model $(X_k,Y_k)$ is known to forget its initial condition as $k\to\infty$ regardless of the observation structure when the signal possesses sufficiently strong…
We study dynamic allocation problems for discrete time multi-armed bandits under uncertainty, based on the the theory of nonlinear expectations. We show that, under strong independence of the bandits and with some relaxation in the…
In this paper, we introduce a new wavelet tool for studying the degree of non-periodicity of time series that is based on some recently defined tools, such as the \textit{windowed scalogram} and the \textit{scale index}. It is especially…
Recent result shows how to compute distributively and efficiently the linear MMSE for the multiuser detection problem, using the Gaussian BP algorithm. In the current work, we extend this construction, and show that operating this algorithm…
The Kalman filter (KF) is used in a variety of applications for computing the posterior distribution of latent states in a state space model. The model requires a linear relationship between states and observations. Extensions to the Kalman…
We propose a method for inference on moderately high-dimensional, nonlinear, non-Gaussian, partially observed Markov process models for which the transition density is not analytically tractable. Markov processes with intractable transition…