Related papers: Smart Finite State Devices: A Modeling Framework f…

In this paper authors are going to present a Markov Decision Process (MDP) based algorithm in Industrial Internet of Things (IIoT) as a safety compliance layer for human in loop system. Though some industries are moving towards Industry 4.0…

Optimal scheduling of deferrable electrical loads can reshape the aggregated load profile to achieve higher operational efficiency and reliability. This paper studies deferrable load scheduling under demand charge that imposes a penalty on…

We consider a class of optimization problems over stochastic variables where the algorithm can learn information about the value of any variable through a series of costly steps; we model this information acquisition process as a Markov…

Demand response has been implemented by distribution system operators to reduce peak demand and mitigate contingency issues on distribution lines and substations. Specifically, the campus based commercial buildings make the major…

We develop a qualitative theory of Markov Decision Processes (MDPs) and Partially Observable MDPs that can be used to model sequential decision making tasks when only qualitative information is available. Our approach is based upon an…

Electricity usage is a major portion of utility bills and the best place to start lowering them. An effective home energy management approach is introduced to decrease customers' electricity bills by determining the optimal appliance…

This paper examines the use of in-store customers as delivery couriers in a centralized crowd-shipping system, targeting the growing need for efficient last-mile delivery in urban areas. We consider a brick-and-mortar retail setting where…

The paper deals with finite-state Markov decision processes (MDPs) with integer weights assigned to each state-action pair. New algorithms are presented to classify end components according to their limiting behavior with respect to the…

Flexibility is a key enabler for the smart grid, required to facilitate Demand Side Management (DSM) programs, managing electrical consumption to reduce peaks, balance renewable generation and provide ancillary services to the grid.…

We consider a single-server queue where interarrival and service times depend linearly and randomly on customer waiting times, and establish a sample-path moderate deviation principle (MDP) for the waiting time process. The waiting times…

Motivated by the post-disaster distribution system restoration problem, in this paper, we study the problem of synthesizing the optimal policy for a Markov Decision Process (MDP) from a sequence of goal sets. For each goal set, our aim is…

Model checking undiscounted reachability and expected-reward properties on Markov decision processes (MDPs) is key for the verification of systems that act under uncertainty. Popular algorithms are policy iteration and variants of value…

This paper proposes a new formulation for the dynamic resource allocation problem, which converts the traditional MDP model with known parameters and no capacity constraints to a new model with uncertain parameters and a resource capacity…

In cyber-physical systems such as automobiles, measurement data from sensor nodes should be delivered to other consumer nodes such as actuators in a regular fashion. But, in practical systems over unreliable media such as wireless, it is a…

In this paper, we present a Distributionally Robust Markov Decision Process (DRMDP) approach for addressing the dynamic epidemic control problem. The Susceptible-Exposed-Infectious-Recovered (SEIR) model is widely used to represent the…

Online learning algorithms are designed to perform in non-stationary environments, but generally there is no notion of a dynamic state to model constraints on current and future actions as a function of past actions. State-based models are…

Multi-objective model predictive control (MOMPC) for fixed point stabilization requires an automated a priori decision-making (DM) mechanism to translate a high-level preference into a single solution. To this aim, we introduce an approach…

By modeling the uncertainty of spinning reserves provided by energy storage with probabilistic constraints, a new optimal scheduling mode is proposed for minimizing the operating costs of an isolated microgrid (MG) by using…

Increasing electricity prices in South Africa and the imminent threat of load shedding due to the overloaded power grid has led to a need for Demand Side Management (DSM) devices like smart grids. For smart grids to perform to their peak,…

Markov decision processes (MDPs) are a canonical model to reason about decision making within a stochastic environment. We study a fundamental class of infinite MDPs: one-counter MDPs (OC-MDPs). They extend finite MDPs via an associated…