Related papers: Sequential detection of Markov targets with trajec…
Markov processes with stochastic resetting towards the origin generically converge towards non-equilibrium steady-states. Long dynamical trajectories can be thus analyzed via the large deviations at Level 2.5 for the joint probability of…
This work tackles the problem of robust zero-shot planning in non-stationary stochastic environments. We study Markov Decision Processes (MDPs) evolving over time and consider Model-Based Reinforcement Learning algorithms in this setting.…
We consider the problem of simultaneous detection and estimation under a sequential framework. In particular we are interested in sequential tests that distinguish between the null and the alternative hypothesis and every time the decision…
We present a hidden Markov model that describes variation in an animal's position associated with varying levels of activity in action potential spike trains of individual place cell neurons. The model incorporates a coarse-graining of…
The distributionally robust Markov Decision Process (MDP) approach asks for a distributionally robust policy that achieves the maximal expected total reward under the most adversarial distribution of uncertain parameters. In this paper, we…
Herein, we analyze an efficient branching particle method for asymptotic solutions to a class of continuous-discrete filtering problems. Suppose that $t\to X_t$ is a Markov process and we wish to calculate the measure-valued process…
Consider the standard stochastic reaction network model where the dynamics is given by a continuous-time Markov chain over a discrete lattice. For such models, estimation of parameter sensitivities is an important problem, but the existing…
This paper considers maximum-a-posteriori (MAP) and linear discriminant based MAP detectors to detect changes in the mean and covariance of a stochastic input, driving specific network nodes, using noisy measurements from sensors…
The number of resident space objects is rising at an alarming rate. Mega-constellations and breakup events are proliferating in most orbital regimes, and safe navigation is becoming increasingly problematic. It is important to be able to…
Sequential change diagnosis is the joint problem of detection and identification of a sudden and unobservable change in the distribution of a random sequence. In this problem, the common probability law of a sequence of i.i.d. random…
This paper considers estimating the parameters in a regime-switching stochastic differential equation(SDE) driven by Normal Inverse Gaussian(NIG) noise. The model under consideration incorporates a continuous-time finite state Markov chain…
This paper has been withdrawn by the authors. Please see arXiv:1302.6058. We consider the sequential joint detection and estimation problem. Minimizing the average stopping time subject to a combination of detection and estimation…
In this paper, we consider the stability analysis of large-scale distributed networked control systems with random communication delays between linearly interconnected subsystems. The stability analysis is performed in the Markov jump…
Periodic recurrence is a prominent behavioural of many biological phenomena, including cell cycle and circadian rhythms. Although deterministic models are commonly used to represent the dynamics of periodic phenomena, it is known that they…
We study one-sided and $\alpha$-correct sequential hypothesis testing for data generated by an ergodic Markov chain. The null hypothesis is that the unknown transition matrix belongs to a prescribed set $P$ of stochastic matrices, and the…
We study detection of random signals corrupted by noise that over time switch their values (states) from a finite set of possible values, where the switchings occur at unknown points in time. We model such signals by means of a random…
We investigate sequential change point estimation and detection in univariate nonparametric settings, where a stream of independent observations from sub-Gaussian distributions with a common variance factor and piecewise-constant but…
Herein, the Hidden Markov Model is expanded to allow for Markov chain observations. In particular, the observations are assumed to be a Markov chain whose one step transition probabilities depend upon the hidden Markov chain. An…
Analysis and synthesis of safety-critical autonomous systems are carried out using models which are often dynamic. Two central features of these dynamic systems are parameters and unmodeled dynamics. This paper addresses the use of a…
This paper compiles several aspects of the dynamics of stochastic approximation algorithms with Markov iterate-dependent noise when the iterates are not known to be stable beforehand. We achieve the same by extending the lock-in probability…