Related papers: Partially Observed Markov Decision Processes. Prob…

Partially observable Markov decision processes (POMDPs) have recently become popular among many AI researchers because they serve as a natural model for planning under uncertainty. Value iteration is a well-known algorithm for finding…

In this document I aim to give an informal treatment of automatic Backward Filtering Forward Guiding, a general algorithm for conditional sampling from a Markov process on a directed acyclic graph. I'll show that the underlying ideas can be…

This paper proposes an accelerated method for approximately solving partially observable Markov decision process (POMDP) problems offline. Our method carefully combines two existing tools: Anderson acceleration (AA) and the fast informed…

This paper provides a comprehensive tutorial on a family of Model Predictive Control (MPC) formulations, known as MPC for tracking, which are characterized by including an artificial reference as part of the decision variables in the…

Today most of the information in all areas is available over the web. It increases the web utilization as well as attracts the interest of researchers to improve the effectiveness of web access and web utilization. As the number of web…

We revisit the problem of maximising the expected length of increasing subsequence that can be selected from a marked Poisson process by an online strategy. Resorting to a natural size variable, the problem is represented in terms of a…

Online solvers for partially observable Markov decision processes have been applied to problems with large discrete state spaces, but continuous state, action, and observation spaces remain a challenge. This paper begins by investigating…

In this work we introduce new approximate similarity relations that are shown to be key for policy (or control) synthesis over general Markov decision processes. The models of interest are discrete-time Markov decision processes, endowed…

Active classification, i.e., the sequential decision-making process aimed at data acquisition for classification purposes, arises naturally in many applications, including medical diagnosis, intrusion detection, and object tracking. In this…

This paper presents a novel algorithm for efficient online estimation of the filter derivatives in general hidden Markov models. The algorithm, which has a linear computational complexity and very limited memory requirements, is furnished…

Markov decision processes (MDPs) in queues and networks have been an interesting topic in many practical areas since the 1960s. This paper provides a detailed overview on this topic and tracks the evolution of many basic results. Also, this…

The contribution gives a micro-structural insight into the pedestrian decision process during an egress situation. A method how to extract the decisions of pedestrians from the trajectories recorded during the experiments is introduced. The…

This paper deals with the optimal stopping problem under partial observation for piecewise-deterministic Markov processes. We first obtain a recursive formulation of the optimal filter process and derive the dynamic programming equation of…

In this paper we propose a model to study the appropriation of knowledge of one student in a non-collaborative online class. We formulate a stochastic model based on the quality of the teacher's class and the affinity of the student to…

Many medical decision-making tasks can be framed as partially observed Markov decision processes (POMDPs). However, prevailing two-stage approaches that first learn a POMDP and then solve it often fail because the model that best fits the…

We introduce Markov substitute processes, a new model at the crossroad of statistics and formal grammars, and prove its main property : Markov substitute processes with a given support form an exponential family.

The program of studying general nonlinear Markov processes was put forward in V. N. Kolokoltsov "Nonlinear Markov Semigroups and Interacting L\'evy Type Processes" (Journ. Stat. Physics 126:3 (2007), 585-642), and was developed by the…

News outlets are now more than ever incentivized to provide their audience with slanted news, while the intrinsic homophilic nature of online social media may exacerbate polarized opinions. Here, we propose a new dynamic latent space model…

Partially observable Markov decision processes (POMDPs) are widely used in probabilistic planning problems in which an agent interacts with an environment using noisy and imprecise sensors. We study a setting in which the sensors are only…

There is much interest in using partially observable Markov decision processes (POMDPs) as a formal model for planning in stochastic domains. This paper is concerned with finding optimal policies for POMDPs. We propose several improvements…