Related papers: Factoring Exogenous State for Model-Free Monte Car…

This work studies a multi-agent Markov decision process (MDP) that can undergo agent dropout and the computation of policies for the post-dropout system based on control and sampling of the pre-dropout system. The central planner's…

Building upon factor decomposition to overcome the curse of dimensionality inherent in multivariate volatility processes, we develop a factor model-based multivariate stochastic volatility (fMSV) framework. We propose a two-stage estimation…

There is a widespread intuition that model-based control methods should be able to surpass the data efficiency of model-free approaches. In this paper we attempt to evaluate this intuition on various challenging locomotion tasks. We take a…

In this brief, a model-free adaptive predictive control (MFAPC) is proposed. It outperforms the current model-free adaptive control (MFAC) for not only solving the time delay problem in multiple-input multiple-output (MIMO) systems but also…

Software-intensive systems, such as software product lines and robotics, utilise Markov decision processes (MDPs) to capture uncertainty and analyse sequential decision-making problems. Despite the usefulness of conventional policy…

Partially observable Markov decision processes (POMDPs) provide an elegant mathematical framework for modeling complex decision and planning problems in stochastic domains in which states of the system are observable only indirectly, via a…

We introduce a functional matrix product state (FMPS) based method for simulating the real-space representation of continuous-variable (CV) quantum computation. This approach efficiently simulates non-Gaussian CV systems by leveraging their…

We consider large-scale Markov decision processes (MDPs) with parameter uncertainty, under the robust MDP paradigm. Previous studies showed that robust MDPs, based on a minimax approach to handle uncertainty, can be solved using dynamic…

An efficient simulation-based methodology is proposed for the rolling window estimation of state space models, called particle rolling Markov chain Monte Carlo (MCMC) with double block sampling. In our method, which is based on Sequential…

Decision-making for engineering systems can be efficiently formulated as a Markov Decision Process (MDP) or a Partially Observable MDP (POMDP). Typical MDP and POMDP solution procedures utilize offline knowledge about the environment and…

Constrained decision-making is essential for designing safe policies in real-world control systems, yet simulated environments often fail to capture real-world adversities. We consider the problem of learning a policy that will maximize the…

Partially observable Markov decision processes (POMDP) are a useful model for decision-making under partial observability and stochastic actions. Partially Observable Monte-Carlo Planning is an online algorithm for deciding on the next…

In this paper, a novel partial form dynamic linearization (PFDL) data-driven model-free adaptive predictive control (MFAPC) method is proposed for a class of discrete-time single-input single-output nonlinear systems. The main contributions…

We model online recommendation systems using the hidden Markov multi-state restless multi-armed bandit problem. To solve this we present Monte Carlo rollout policy. We illustrate numerically that Monte Carlo rollout policy performs better…

Model-based reinforcement learning approaches leverage a forward dynamics model to support planning and decision making, which, however, may fail catastrophically if the model is inaccurate. Although there are several existing methods…

Markov Chain Monte Carlo (MCMC) sampling is computationally expensive, especially for complex models. Alternative methods make simplifying assumptions about the posterior to reduce computational burden, but their impact on predictive…

Although histogram methods have been extremely effective for analyzing data from Monte Carlo simulations, they do have certain limitations, including the range over which they are valid and the difficulties of combining data from…

Model-Free Control (MFC) has been applied to a wide variety of systems in which it has shown its performance. MFC offers "model-free operation", but the controller design requires some information from the nominal plant. This paper…

Markov decision processes (MDPs) are a popular model for performance analysis and optimization of stochastic systems. The parameters of stochastic behavior of MDPs are estimates from empirical observations of a system; their values are not…

The continuous nature of belief states in POMDPs presents significant computational challenges in learning the optimal policy. In this paper, we consider an approach that solves a Partially Observable Reinforcement Learning (PORL) problem…