Related papers: On Incorporating Forecasts into Linear State Space…

A fundamental (and largely open) challenge in sequential decision-making is dealing with non-stationary environments, where exogenous environmental conditions change over time. Such problems are traditionally modeled as non-stationary…

We consider Markov models of stochastic processes where the next-step conditional distribution is defined by a kernel density estimator (KDE), similar to Markov forecast densities and certain time-series bootstrap schemes. The KDE Markov…

Effectively modeling non-stationary dynamics in probabilistic multivariate time series(MTS) forecasting requires balancing expressiveness with robustness. Existing parametric approaches benefit from strong inductive biases but lack…

As we deploy autonomous agents in safety-critical domains, it becomes important to develop an understanding of their internal mechanisms and representations. We outline an approach to imitation learning for reverse-engineering black box…

We show that nonequilibrium dynamics can play a constructive role in unsupervised machine learning by inducing the spontaneous emergence of latent-state cycles. We introduce a model in which visible and hidden variables interact through two…

Hourly maxima of 3-second wind gust speeds are prominent indicators of the severity of wind storms, and accurately forecasting them is thus essential for populations, civil authorities and insurance companies. Space-time max-stable models…

State-space systems encompass a broad class of algorithms used for modeling and forecasting time series. For such systems to be effective, two objectives must be met: (i) accurate point forecasts of the time series must be produced, and…

Markov decisions processes (MDPs) are becoming increasing popular as models of decision theoretic planning. While traditional dynamic programming methods perform well for problems with small state spaces, structured methods are needed for…

This paper is concerned with a data-driven technique for constructing finite Markov decision processes (MDPs) as finite abstractions of discrete-time stochastic control systems with unknown dynamics while providing formal closeness…

The use of algorithmic predictions in decision-making leads to a feedback loop where the models we deploy actively influence the data distributions we see, and later use to retrain on. This dynamic was formalized by Perdomo et al. 2020 in…

Many scientific and engineering systems exhibit intrinsically multimodal behavior arising from latent regime switching and non-unique physical mechanisms. In such settings, learning the full conditional distribution of admissible outcomes…

We adopt a policy optimization viewpoint towards policy evaluation for robust Markov decision process with $\mathrm{s}$-rectangular ambiguity sets. The developed method, named first-order policy evaluation (FRPE), provides the first unified…

Predicting the demand for electricity with uncertainty helps in planning and operation of the grid to provide reliable supply of power to the consumers. Machine learning (ML)-based demand forecasting approaches can be categorized into (1)…

In the realm of control systems, model predictive control (MPC) has exhibited remarkable potential; however, its reliance on accurate models and substantial computational resources has hindered its broader application, especially within…

New theoretical results are presented here on the recently introduced model called mixed states MRF. Such models were introduced in the context of image motion analysis and are useful to represent information which can take both discrete…

Modeling event dynamics is central to many disciplines. Patterns in observed event arrival times are commonly modeled using point processes. Such event arrival data often exhibits self-exciting, heterogeneous and sporadic trends, which is…

In this work, we study dynamic programming (DP) algorithms for partially observable Markov decision processes with jointly continuous and discrete state-spaces. We consider a class of stochastic systems which have coupled discrete and…

Using a novel toy nautical navigation environment, we show that dynamic programming can be used when only incomplete information about a partially observed Markov decision process (POMDP) is known. By incorporating uncertainty into our…

Graph games and Markov decision processes (MDPs) are standard models in reactive synthesis and verification of probabilistic systems with nondeterminism. The class of $\omega$-regular winning conditions; e.g., safety, reachability,…

This work proposes a compositional data-driven technique for the construction of finite Markov decision processes (MDPs) for large-scale stochastic networks with unknown mathematical models. Our proposed framework leverages dissipativity…