Related papers: Active Classification for POMDPs: a Kalman-like St…
We study the problem of synthesizing a controller that maximizes the entropy of a partially observable Markov decision process (POMDP) subject to a constraint on the expected total reward. Such a controller minimizes the predictability of a…
We consider the problem of optimal distributed beamforming in a sensor network where the sensors observe a dynamic parameter in noise and coherently amplify and forward their observations to a fusion center (FC). The FC uses a Kalman filter…
We consider state and parameter estimation for a dynamical system having both time-varying and time-invariant parameters. It has been shown that the robustness of the Markov Chain Monte Carlo (MCMC) algorithm for estimating time-invariant…
We consider the problem belief-state monitoring for the purposes of implementing a policy for a partially-observable Markov decision process (POMDP), specifically how one might approximate the belief state. Other schemes for belief-state…
This paper considers the Linear Minimum Variance recursive state estimation for the linear discrete time dynamic system with random state transition and measurement matrices, i.e., random parameter matrices Kalman filtering. It is shown…
This paper deals with the question of how to most effectively conduct experiments in Partially Observed Markov Decision Processes so as to provide data that is most informative about a parameter of interest. Methods from Markov decision…
This paper proposes a joint input and state dynamic estimation scheme for power networks in microgrids and active distribution systems with unknown inputs. The conventional dynamic state estimation of power networks in the transmission…
We study reinforcement learning problems where state observations are stochastically triggered by actions, a constraint common in many real-world applications. This framework is formulated as Action-Triggered Sporadically Traceable Markov…
We explore Markov-modulated marked Poisson processes (MMMPPs) as a natural framework for modelling patients' disease dynamics over time based on medical claims data. In claims data, observations do not only occur at random points in time…
We consider optimal signalling and control of discrete-time nonlinear partially observable stochastic systems in state space form. In the first part of the paper, we characterize the operational {\it control-coding capacity}, $C_{FB}$ in…
This work addresses stochastic optimal control problems where the unknown state evolves in continuous time while partial, noisy, and possibly controllable measurements are only available in discrete time. We develop a framework for…
Optimal decision-making presents a significant challenge for autonomous systems operating in uncertain, stochastic and time-varying environments. Environmental variability over time can significantly impact the system's optimal decision…
Markov state models (MSMs) have been successful in computing metastable states, slow relaxation timescales and associated structural changes, and stationary or kinetic experimental observables of complex molecules from large amounts of…
In this paper, we propose a class of efficient, accurate, and general methods for solving state-estimation problems with equality and inequality constraints. The methods are based on recent developments in variable splitting and partially…
In this paper, we study the design of an optimal transmission policy for remote state estimation over packet-dropping wireless channels with imperfect channel state information. A smart sensor uses a Kalman filter to estimate the system…
Utilizing highly synchronized measurements from synchrophasors, dynamic state estimation (DSE) can be applied for real-time monitoring of smart grids. Concurrent DSE studies for power systems are intolerant to unknown inputs and potential…
This paper deals with the problem of remote estimation of the state of a discrete-time stochastic linear system observed by a sensor with computational capacity to calculate local estimates. We design an event-triggered communication (ETC)…
This paper studies the optimal control problem for discrete-time nonlinear systems and an approximate dynamic programming-based Model Predictive Control (MPC) scheme is proposed for minimizing a quadratic performance measure. In the…
Designing efficient and rigorous numerical methods for sequential decision-making under uncertainty is a difficult problem that arises in many applications frameworks. In this paper we focus on the numerical solution of a subclass of…
This paper investigates goal-oriented remote monitoring of an unobservable Markov source using energy-harvesting sensors that communicate with a mobile receiver, such as a Low Earth Orbit (LEO) satellite or Unmanned Aerial Vehicle (UAV).…