Related papers: Piecewise deterministic Markov process for conditi…
We present a numerical method to compute the optimal maintenance time for a complex dynamic system applied to an example of maintenance of a metallic structure subject to corrosion. An arbitrarily early intervention may be uselessly costly,…
This article provides an introductory tutorial on structural results in partially observed Markov decision processes (POMDPs). Typically, computing the optimal policy of a POMDP is computationally intractable. We use lattice program- ming…
Repair mechanisms are important within resilient systems to maintain the system in an operational state after an error occurred. Usually, constraints on the repair mechanisms are imposed, e.g., concerning the time or resources required…
This paper deals with control of partially observable discrete-time stochastic systems. It introduces and studies Markov Decision Processes with Incomplete Information and with semi-uniform Feller transition probabilities. The important…
We define a model of a system that deteriorate as a result of (i) shocks, modeled as a compound Poisson process and (ii) deterministic, state dependent progressive rate, with variable and fixed maintenance cost. We define maintenance…
This paper presents with justifications a technique that is useful for the study of piecewise deterministic Markov decision processes (PDMDPs) with general policies and unbounded transition intensities. This technique produces an auxiliary…
Here, we explore the problem of error propagation mitigation in modular digital twins as a sequential decision process. Building on a companion study that used a Hidden Markov Model (HMM) to infer latent error regimes from surrogate-physics…
This paper deals with unconstrained discounted continuous-time Markov decision processes in Borel state and action spaces. Under some conditions imposed on the primitives, allowing unbounded transition rates and unbounded (from both above…
In this paper, we consider a piecewise deterministic Markov process (PDMP), with known flow and deterministic transition measure, and unknown jump rate $\lambda$. To estimate nonparametrically the jump rate, we first construct an adaptive…
It is well known that for any finite state Markov decision process (MDP) there is a memoryless deterministic policy that maximizes the expected reward. For partially observable Markov decision processes (POMDPs), optimal memoryless policies…
This paper presents a novel approach to pricing American options using piecewise diffusion Markov processes (PDifMPs), a type of generalised stochastic hybrid system that integrates continuous dynamics with discrete jump processes. Standard…
A piecewise-deterministic Markov process is a stochastic process whose behavior is governed by an ordinary differential equation punctuated by random jumps occurring at random times. We focus on the nonparametric estimation problem of the…
There is much interest in using partially observable Markov decision processes (POMDPs) as a formal model for planning in stochastic domains. This paper is concerned with finding optimal policies for POMDPs. We propose several improvements…
Implementing an appropriate maintenance policy would help us to have a more reliable system and reduce the total costs. In this paper, a dynamic maintenance plan is proposed for repairable multi-component systems, where each component is…
We consider a class of sequential decision-making problems under uncertainty that can encompass various types of supervised learning concepts. These problems have a completely observed state process and a partially observed modulation…
The topics treated in this thesis are inherently two-fold. The first part considers the problem of a market maker optimally setting bid/ask quotes over a finite time horizon, to maximize her expected utility. The intensities of the orders…
Autonomous agents are limited in their ability to observe the world state. Partially observable Markov decision processes (POMDPs) formally model the problem of planning under world state uncertainty, but POMDPs with continuous actions and…
Machine maintenance is a challenging operational problem, where the goal is to plan sufficient preventive maintenance to avoid machine failures and overhauls. Maintenance is often imperfect in reality and does not make the asset as good as…
As the society becomes more dependent on the presence of electricity, the resilience of the power systems gains more importance. This paper develops a decision support method for distribution system operators to restore electricity after an…
This paper presents a numerical method to calculate the value function for a general discounted impulse control problem for piecewise deterministic Markov processes. Our approach is based on a quantization technique for the underlying…