Related papers: Piecewise deterministic Markov process for conditi…
Using the tools of the Markov Decision Processes, we justify the dynamic programming approach to the optimal impulse control of deterministic dynamical systems. We prove the equivalence of the integral and differential forms of the…
Probabilistic model checking can provide formal guarantees on the behavior of stochastic models relating to a wide range of quantitative properties, such as runtime, energy consumption or cost. But decision making is typically with respect…
We consider piecewise-deterministic optimal control problems in which the environment randomly switches among several deterministic modes, and the goal is to optimize the expected cost up to the termination while taking the likelihood of…
We present a numerical method to compute the survival function and the moments of the exit time for a piecewise-deterministic Markov process (PDMP). Our approach is based on the quantization of an underlying discrete-time Markov chain…
This paper investigates MDPs with intermittent state information. We consider a scenario where the controller perceives the state information of the process via an unreliable communication channel. The transmissions of state information…
Predictive maintenance (PdM) has become a crucial element of modern industrial practice. PdM plays a significant role in operational dependability and cost management by decreasing unforeseen downtime and optimizing asset life cycle…
Structural Health Monitoring (SHM) describes a process for inferring quantifiable metrics of structural condition, which can serve as input to support decisions on the operation and maintenance of infrastructure assets. Given the long…
We consider risk-sensitive Markov decision processes (MDPs), where the MDP model is influenced by a parameter which takes values in a compact metric space. We identify sufficient conditions under which small perturbations in the model…
In this paper, we propose a novel class of Piecewise Deterministic Markov Processes (PDMPs) that are designed to sample from probability distributions $\pi$ supported on a convex set $\mathcal{M}$. This class of PDMPs adapts the concept of…
This tutorial describes recently developed general optimality conditions for Markov Decision Processes that have significant applications to inventory control. In particular, these conditions imply the validity of optimality equations and…
Several aspects influence corrosive processes in RC structures, such as environmental conditions, structural geometry, and mechanical properties. Since these aspects present large randomnesses, probabilistic models allow a more accurate…
Inference on modern Bayesian Neural Networks (BNNs) often relies on a variational inference treatment, imposing violated assumptions of independence and the form of the posterior. Traditional MCMC approaches avoid these assumptions at the…
Markov Decision Processes (MDPs) are stochastic optimization problems that model situations where a decision maker controls a system based on its state. Partially observed Markov decision processes (POMDPs) are generalizations of MDPs where…
This paper addresses the problem of determining the optimal time for an individual to convert retirement savings into a lifetime annuity. The individual invests their wealth into a dividend-paying fund that follows the dynamics of a…
The main goal of this paper is to apply the so-called policy iteration algorithm (PIA) for the long run average continuous control problem of piecewise deterministic Markov processes (PDMP's) taking values in a general Borel space and with…
An optimal control for a dynamical system optimizes a certain objective function. Here we consider the construction of an optimal control for a stochastic dynamical system with a random structure, Poisson perturbations and random jumps,…
The reliability of a system can be improved by the addition of redundant elements, giving rise to the well-known redundancy allocation problem (RAP). We propose a novel extension to the RAP called the Bi-Objective Integrated Design and…
We study stochastic motion planning problems which involve a controlled process, with possibly discontinuous sample paths, visiting certain subsets of the state-space while avoiding others in a sequential fashion. For this purpose, we first…
Planning under uncertainty is critical to robotics. The Partially Observable Markov Decision Process (POMDP) is a mathematical framework for such planning problems. It is powerful due to its careful quantification of the non-deterministic…
In this paper, we consider the problem of optimizing the worst-case behavior of a partially observed system. All uncontrolled disturbances are modeled as finite-valued uncertain variables. Using the theory of cost distributions, we present…