Related papers: A Markov Decision Process Framework for Enhancing …
Electric power networks are critical lifelines, and their disruption during earthquakes can lead to severe cascading failures and significantly hinder post-disaster recovery. Enhancing their seismic resilience requires identifying and…
We consider online reinforcement learning in episodic Markov decision process (MDP) with unknown transition function and stochastic rewards drawn from some fixed but unknown distribution. The learner aims to learn the optimal policy and…
We develop a structure-aware reinforcement learning (RL) approach for delay- and energy-aware flow allocation in 5G User Plane Functions (UPFs). We consider a dynamic system with $K$ heterogeneous UPFs of varying capacities that handle…
We consider a risk-sensitive optimization of consumption-utility on infinite time horizon where the one-period investment gain depends on an underlying economic state whose evolution over time is assumed to be described by a discrete-time,…
Markov decision processes (MDP) are a well-established model for sequential decision-making in the presence of probabilities. In robust MDP (RMDP), every action is associated with an uncertainty set of probability distributions, modelling…
Existing power cyber-physical system (CPS) risk prediction results are inaccurate as they fail to reflect the actual physical characteristics of the components and the specific operational status. A new method based on dependent Markov…
Cloud infrastructures are being increasingly utilized in critical infrastructures such as banking/finance, transportation and utility management. Sophistication and resources used in recent security breaches including those on critical…
A precise estimation of the Rate of Change of Frequency (RoCoF) is crucial for secure power system operation. In fact, RoCoF is strictly related to the amount of the available physical and/or virtual inertia of the system and the severity…
Critical infrastructure increasingly relies on interconnected cyber-physical systems whose security incidents can escalate rapidly into safety and operational failures. Existing decision-support approaches struggle to support real-time…
We consider the problem of energy-efficient on-line scheduling for slice-parallel video decoders on multicore systems. We assume that each of the processors are Dynamic Voltage Frequency Scaling (DVFS) enabled such that they can…
A Markov decision process-based state switching is devised, implemented, and analyzed for proximity operations of various autonomous vehicles. The framework contains a pose estimator along with a multi-state guidance algorithm. The unified…
In this paper, we consider the problem of safety assessment for Markov decision processes without explicit knowledge of the model. We aim to learn probabilistic safety specifications associated with a given policy without compromising the…
Damage scenarios for large networks are considered. The cascade scenario is described by means of powers of adjacency matrix. More difficult probabilistic variants of the large network damage are modeling by Markov chains. For reliability…
This paper provides an enhanced modeling of the contingency response that collectively reflects high-fidelity physical and operational characteristics of power grids. Integrating active and reactive power contingency responses into the…
Repair mechanisms are important within resilient systems to maintain the system in an operational state after an error occurred. Usually, constraints on the repair mechanisms are imposed, e.g., concerning the time or resources required…
Distribution networks are increasingly exposed to threats such as extreme weather, aging infrastructure, and cyber risks--resulting in more frequent contingencies and outages, a trend likely to persist. Microgrids, particularly dynamic…
Battery sizing and siting problems are computationally challenging due to the need to make long-term planning decisions that are cognizant of short-term operational decisions. This paper considers sizing, siting, and operating batteries in…
Flexible district heating grids form an important part of future, low-carbon energy systems. We examine probabilistic state estimation in such grids, i.e., we aim to estimate the posterior probability distribution over all grid state…
In the electrical grid, the distribution system is themost vulnerable to severe weather events. Well-placed and coordinatedupgrades, such as the combination of microgrids, systemhardening and additional line redundancy, can greatly reduce…
Wildfires are natural disasters capable of damaging economies and communities. When wildfires become uncontrollable, Incident Manager Teams (IMT's) dispatch response vehicles to key assets to undertake protective tasks and so mitigate the…