Related papers: Partially Observed Markov Decision Processes. Prob…

Noisy sensing, imperfect control, and environment changes are defining characteristics of many real-world robot tasks. The partially observable Markov decision process (POMDP) provides a principled mathematical framework for modeling and…

In this paper we consider a control problem for a Partially Observable Piecewise Deterministic Markov Process of the following type: After the jump of the process the controller receives a noisy signal about the state and the aim is to…

Partially observable Markov decision processes (POMDPs) provide an elegant mathematical framework for modeling complex decision and planning problems in stochastic domains in which states of the system are observable only indirectly, via a…

Understanding how people allocate visual attention is central to Human-Computer Interaction (HCI), yet existing computational models of attention are often either descriptive, task-specific, or difficult to interpret. My dissertation…

POMDPs are useful models for systems where the true underlying state is not known completely to an outside observer; the outside observer incompletely knows the true state of the system, and observes a noisy version of the true system…

Planning under uncertainty is critical to robotics. The Partially Observable Markov Decision Process (POMDP) is a mathematical framework for such planning problems. It is powerful due to its careful quantification of the non-deterministic…

Markov decision processes continue to gain in popularity for modeling a wide range of applications ranging from analysis of supply chains and queuing networks to cognitive science and control of autonomous vehicles. Nonetheless, they tend…

This is a complementary document for the paper titled "Asynchronous Distributed Gaussian Process Regression for Online Learning and Dynamical Systems".

We revisit the problem of estimating the parameters of a partially observed diffusion process, consisting of a hidden state process and an observed process, with a continuous time parameter. The estimation is to be done online, i.e. the…

We consider multiple parallel Markov decision processes (MDPs) coupled by global constraints, where the time varying objective and constraint functions can only be observed after the decision is made. Special attention is given to how well…

In this paper, some points to the convergence analysis performed in the paper [A new computing approach for power signal modeling using fractional adaptive algorithms, ISA Transactions 68 (2017) 189-202] are presented. It is highlighted…

This is the first part of a possible monograph on the duality of Markov processes. It contains a proof of Fitzsimmons' existence theorem of a moderate Markov dual process relative to an excessive measure, m, together with the necessary…

The online Markov decision process (MDP) is a generalization of the classical Markov decision process that incorporates changing reward functions. In this paper, we propose practical online MDP algorithms with policy iteration and…

General purpose intelligent learning agents cycle through (complex,non-MDP) sequences of observations, actions, and rewards. On the other hand, reinforcement learning is well-developed for small finite state Markov Decision Processes…

Filtering is concerned with the sequential estimation of the state, and uncertainties, of a Markovian system, given noisy observations. It is particularly difficult to achieve accurate filtering in complex dynamical systems, such as those…

The ongoing rapid expansion of the Internet greatly increases the necessity of effective recommender systems for filtering the abundant information. Extensive research for recommender systems is conducted by a broad range of communities…

In many engineering systems, proper predictive maintenance and operational control are essential to increase efficiency and reliability while reducing maintenance costs. However, one of the major challenges is that many sensors are used for…

Markov processes are popular mathematical models, studied by theoreticians for their intriguing properties, and applied by practitioners for their flexible structure. With this book we teach how to model and analyze Markov processes. We…

In this paper, we investigate the problem of remote estimation of a discrete-time joint Markov process using multiple sensors. Each sensor observes a different component of the joint Markov process, and in each time slot, the monitor…

Partially Observable Markov Decision Processes (POMDPs) provide a rich framework for sequential decision-making under uncertainty in stochastic domains. However, solving a POMDP is often intractable except for small problems due to their…