Related papers: Smart Finite State Devices: A Modeling Framework f…
We consider non-standard Markov Decision Processes (MDPs) where the target function is not only a simple expectation of the accumulated reward. Instead, we consider rather general functionals of the joint distribution of terminal state and…
The expansion of residential demand response programs and increased deployment of controllable loads will require accurate appliance-level load modeling and forecasting. This paper proposes a conditional hidden semi-Markov model to describe…
Electric power network expanded rapidly in recent decades due of the excessive need of electricity in every aspect of life, including critical infrastructures such as medical services, and transportation and communication systems. Natural…
We propose two scheduling algorithms that seek to optimize the quality of scalably coded videos that have been stored at a video server before transmission.} The first scheduling algorithm is derived from a Markov Decision Process (MDP)…
Modern recommender systems usually present items as a streaming, one-dimensional ranking list. Recently there is a trend in e-commerce that the recommended items are organized grid-based panels with two dimensions where users can view the…
Factored Markov decision processes (MDPs) are a prominent paradigm within the artificial intelligence community for modeling and solving large-scale MDPs whose rewards and dynamics decompose into smaller, loosely interacting components.…
Energy Markov Decision Processes (EMDPs) are finite-state Markov decision processes where each transition is assigned an integer counter update and a rational payoff. An EMDP configuration is a pair s(n), where s is a control state and n is…
In piecewise-deterministic Markov processes (PDMPs) the state of a finite-dimensional system evolves continuously, but the evolutive equation may change randomly as a result of discrete switches. A running cost is integrated along the…
Action delays degrade the performance of reinforcement learning in many real-world systems. This paper proposes a formal definition of delay-aware Markov Decision Process and proves it can be transformed into standard MDP with augmented…
The integration of renewable sources poses challenges at the operational and economic levels of the power grid. In terms of keeping the balance between supply and demand, the usual scheme of supply following load may not be appropriate for…
A well-performing prediction model is vital for a recommendation system suggesting actions for energy-efficient consumer behavior. However, reliable and accurate predictions depend on informative features and a suitable model design to…
Robust Markov Decision Processes (MDPs) are receiving much attention in learning a robust policy which is less sensitive to environment changes. There are an increasing number of works analyzing sample-efficiency of robust MDPs. However,…
In this work, a versatile mathematical framework for multi-state probabilistic modeling of Resistive Switching (RS) devices is proposed for the first time. The mathematical formulation of memristor and Markov jump processes are combined…
We consider the problem of statistical inference in a parametric finite Markov chain model and develop a robust estimator of the parameters defining the transition probabilities via minimization of a suitable (empirical) version of the…
We introduce Multi-Environment Markov Decision Processes (MEMDPs) which are MDPs with a set of probabilistic transition functions. The goal in a MEMDP is to synthesize a single controller with guaranteed performances against all…
Markov decision processes (MDPs) are a popular model for performance analysis and optimization of stochastic systems. The parameters of stochastic behavior of MDPs are estimates from empirical observations of a system; their values are not…
We propose and solve a stochastic dynamic programming (DP) problem addressing the optimal provision of regulation service reserves (RSR) by controlling dynamic demand preferences in smart buildings. A major contribution over past dynamic…
This paper studies optimal market making for large-tick assets in the presence of latency. We consider a random walk model for the asset price, and formulate the market maker's optimization problem using Markov Decision Processes (MDP). We…
The evolution of the power grid towards the so-called Smart Grid, where information technologies help improve the efficiency of electricity production, distribution and consumption, allows to use the fine-grained control brought by the…
In this paper, we propose a realistic multiple dynamic pricing approach to demand response in the retail market. First, an adaptive clustering-based customer segmentation framework is proposed to categorize customers into different groups…