Related papers: Adaptive power method for estimating large deviati…
Message-passing architectures struggle to sufficiently model long-range dependencies in node and graph prediction tasks. We propose a novel approach exploiting hierarchical graph structures and adaptive random walks to address this…
We propose a method to sample stationary properties of solutions of stochastic differential equations, which is accurate and efficient if there are rarely visited regions or rare transitions between distinct regions of the state space. The…
Stochastic Gradient Descent (SGD) and its variants are the most used algorithms in machine learning applications. In particular, SGD with adaptive learning rates and momentum is the industry standard to train deep networks. Despite the…
Sampling is an important tool for estimating large, complex sums and integrals over high dimensional spaces. For instance, important sampling has been used as an alternative to exact methods for inference in belief networks. Ideally, we…
This paper investigates the optimization problem of an infinite stage discrete time Markov decision process (MDP) with a long-run average metric considering both mean and variance of rewards together. Such performance metric is important…
We consider an energy storage problem involving a wind farm with a forecasted power output, a stochastic load, an energy storage device, and a connection to the larger power grid with stochastic prices. Electricity prices and wind power…
We present on-line policy gradient algorithms for computing the locally optimal policy of a constrained, average cost, finite state Markov Decision Process. The stochastic approximation algorithms require estimation of the gradient of the…
We develop a stochastic approximation-type algorithm to solve finite state/action, infinite-horizon, risk-aware Markov decision processes. Our algorithm has two loops. The inner loop computes the risk by solving a stochastic saddle-point…
We study continuous time Markov processes on graphs. The notion of frequency is introduced, which serves well as a scaling factor between any Markov time of a continuous time Markov process and that of its jump chain. As an application, we…
Markov chains are simple yet powerful mathematical structures to model temporally dependent processes. They generally assume stationary data, i.e., fixed transition probabilities between observations/states. However, live, real-world…
We show how to calculate the likelihood of dynamical large deviations using evolutionary reinforcement learning. An agent, a stochastic model, propagates a continuous-time Monte Carlo trajectory and receives a reward conditioned upon the…
The aim of this paper is to propose diffusion strategies for distributed estimation over adaptive networks, assuming the presence of spatially correlated measurements distributed according to a Gaussian Markov random field (GMRF) model. The…
This paper presents a novel centralized, variational data assimilation approach for calibrating transient dynamic models in electrical power systems, focusing on load model parameters. With the increasing importance of inverter-based…
We consider the boundary driven harmonic model, i.e. the Markov process associated to the open integrable XXX chain with non-compact spins. Using the factorial moments we characterize the stationary measure as a mixture of product measures.…
We propose using an adaptive sampling method to detect changes for a system with multiple lines. The adaptive sampling utilizes the information in responses to learn on which line is more likely to have a change thus allocating more units…
This paper revisits the temporal difference (TD) learning algorithm for the policy evaluation tasks in reinforcement learning. Typically, the performance of TD(0) and TD($\lambda$) is very sensitive to the choice of stepsizes. Oftentimes,…
In this paper, we describe a stochastic adaptive fast gradient descent method based on the mirror variant of similar triangles method. To our knowledge, this is the first attempt to use adaptivity in stochastic method. Additionally, a main…
In this paper, we study a class of stochastic processes, called evolving network Markov chains, in evolving networks. Our approach is to transform the degree distribution problem of an evolving network to a corresponding problem of evolving…
This thesis is devoted to the study of extreme value statistics in stochastic processes and their applications. In the first part, we obtain exact analytical results on the extreme value statistics of both discrete-time and continuous-time…
We introduce a variational algorithm to estimate the likelihood of a rare event within a nonequilibrium molecular dynamics simulation through the evaluation of an optimal control force. Optimization of a control force within a chosen basis…