Related papers: Asymptotic Stability of the optimal filter for non…
We consider parameter estimations with probes being the boundary driven/dissipated non- equilibrium steady states of XXZ spin 1/2 chains. The parameters to be estimated are the dissipation coupling and the anisotropy of the spin-spin…
Given a stochastic nonlinear system controlled over a possibly noisy communication channel, the paper studies the largest class of channels for which there exist coding and control policies so that the closed-loop system is stochastically…
Likelihood-based inference in stochastic non-linear dynamical systems, such as those found in chemical reaction networks and biological clock systems, is inherently complex and has largely been limited to small and unrealistically simple…
Dynamic and evolving operational and economic environments present significant challenges for decision-making. We explore a simulation optimization problem characterized by non-stationary input distributions with regime-switching dynamics…
We study Bayesian optimal control of a general class of smoothly parameterized Markov decision problems. Since computing the optimal control is computationally expensive, we design an algorithm that trades off performance for computational…
We consider periodic Markov chains with absorption. Applying to iterates of this periodic Markov chain criteria for the exponential convergence of conditional distributions of aperiodic absorbed Markov chains, we obtain exponential…
This paper discusses the problem of estimating a stochastic signal from nonlinear uncertain observations with time-correlated additive noise described by a first-order Markov process. Random deception attacks are assumed to be launched by…
A Markov-switching observation-driven model is a stochastic process $((S_t,Y_t))_{t \in \mathbb{Z}}$ where $(S_t)_{t \in \mathbb{Z}}$ is an unobserved Markov chain on a finite set and $(Y_t)_{t \in \mathbb{Z}}$ is an observed stochastic…
The nonlinear filter associated with the discrete time signal-observation model $(X_k,Y_k)$ is known to forget its initial condition as $k\to\infty$ regardless of the observation structure when the signal possesses sufficiently strong…
For a Markov transition kernel $P$ and a probability distribution $ \mu$ on nonnegative integers, a time-sampled Markov chain evolves according to the transition kernel $P_{\mu} = \sum_k \mu(k)P^k.$ In this note we obtain CLT conditions for…
The asymptotic normality in multi-dimension of the nonparametric estimator of the transition probabilities of a Markov renewal chain is proved, and is applied to that of other nonparametric estimators involved with the associated…
We consider a non-linear filtering problem, whereby the signal obeys the stochastic Navier-Stokes equations and is observed through a linear mapping with additive noise. The setup is relevant to data assimilation for numerical weather…
Herein, we analyze an efficient branching particle method for asymptotic solutions to a class of continuous-discrete filtering problems. Suppose that $t\to X_t$ is a Markov process and we wish to calculate the measure-valued process…
We establish conditions for an exponential rate of forgetting of the initial distribution of nonlinear filters in $V$-norm, path-wise along almost all observation sequences. In contrast to previous works, our results allow for unbounded…
Filtering is concerned with the sequential estimation of the state, and uncertainties, of a Markovian system, given noisy observations. It is particularly difficult to achieve accurate filtering in complex dynamical systems, such as those…
In this paper, we investigate a nonparametric approach to provide a recursive estimator of the transition density of a non-stationary piecewise-deterministic Markov process, from only one observation of the path within a long time. In this…
Nonparametric identification and maximum likelihood estimation for finite-state hidden Markov models are investigated. We obtain identification of the parameters as well as the order of the Markov chain if the transition probability…
Markov Decision Processes (MDPs) are a popular class of models suitable for solving control decision problems in probabilistic reactive systems. We consider parametric MDPs (pMDPs) that include parameters in some of the transition…
We consider the problems of identifying and estimating dynamical parameters of an ergodic quantum Markov chain, when only the stationary output is accessible for measurements. On the identifiability question, we show that the knowledge of…
This paper studies Markov Decision Processes under parameter uncertainty. We adapt the distributionally robust optimization framework, and assume that the uncertain parameters are random variables following an unknown distribution, and…