Related papers: MDP based Decision Support for Earthquake Damaged …

When human operators of cyber-physical systems encounter surprising behavior, they often consider multiple hypotheses that might explain it. In some cases, taking information-gathering actions such as additional measurements or control…

Markov decision processes (MDPs) are used to model a wide variety of applications ranging from game playing over robotics to finance. Their optimal policy typically maximizes the expected sum of rewards given at each step of the decision…

We consider risk-sensitive Markov decision processes (MDPs), where the MDP model is influenced by a parameter which takes values in a compact metric space. We identify sufficient conditions under which small perturbations in the model…

This paper investigates MDPs with intermittent state information. We consider a scenario where the controller perceives the state information of the process via an unreliable communication channel. The transmissions of state information…

In this work, we study the problem of actively classifying the attributes of dynamical systems characterized as a finite set of Markov decision process (MDP) models. We are interested in finding strategies that actively interact with the…

Value iteration is a well-known method of solving Markov Decision Processes (MDPs) that is simple to implement and boasts strong theoretical convergence guarantees. However, the computational cost of value iteration quickly becomes…

We study minimax optimal reinforcement learning in episodic factored Markov decision processes (FMDPs), which are MDPs with conditionally independent transition components. Assuming the factorization is known, we propose two model-based…

Civil and maritime engineering systems, among others, from bridges to offshore platforms and wind turbines, must be efficiently managed as they are exposed to deterioration mechanisms throughout their operational life, such as fatigue or…

The lack of a comprehensive decision-making approach at the community level is an important problem that warrants immediate attention. Network-level decision-making algorithms need to solve large-scale optimization problems that pose…

Markov decision processes (MDPs) are a standard model for sequential decision-making problems and are widely used across many scientific areas, including formal methods and artificial intelligence (AI). MDPs do, however, come with the…

Dispatching mobile resources such as repair crews and mobile emergency generators is essential for the rapid restoration of distribution systems after extreme events. However, the restoration process is affected by various uncertain factors…

The frequency of extreme events (e.g., hurricanes, earthquakes, and floods) and man-made attacks (cyber and physical attacks) has increased dramatically in recent years. These events have severely impacted power systems ranging from long…

This paper proposes an optimization strategy to assist utility operators to recover power distribution systems after large outages. Specifically, a mixed-integer linear programming (MILP) model is developed for co-optimizing crews,…

We consider large-scale Markov decision processes (MDPs) with parameter uncertainty, under the robust MDP paradigm. Previous studies showed that robust MDPs, based on a minimax approach to handle uncertainty, can be solved using dynamic…

This paper develops risk-averse models to support system operators in planning and operating the electricity grid under uncertainty from renewable power generation. We incorporate financial risk hedging using conditional value at risk…

We consider synthesis of control policies that maximize the probability of satisfying given temporal logic specifications in unknown, stochastic environments. We model the interaction between the system and its environment as a Markov…

Markov decision processes (MDPs) are a popular model for decision-making in the presence of uncertainty. The conventional view of MDPs in verification treats them as state transformers with probabilities defined over sequences of states and…

Markov decision processes (MDPs) with rewards are a widespread and well-studied model for systems that make both probabilistic and nondeterministic choices. A fundamental result about MDPs is that their minimal and maximal expected rewards…

Developing optimized restoration strategies for power distribution systems (PDSs) is critical to enhancing resilience. Prior knowledge of customer interruption cost (CIC) and load restoration behaviors, particularly cold load pickup (CLPU),…

Decision-making for engineering systems can be efficiently formulated as a Markov Decision Process (MDP) or a Partially Observable MDP (POMDP). Typical MDP and POMDP solution procedures utilize offline knowledge about the environment and…