Related papers: Non-Markovian Optimal Prediction
This article provides an introductory tutorial on structural results in partially observed Markov decision processes (POMDPs). Typically, computing the optimal policy of a POMDP is computationally intractable. We use lattice program- ming…
In the field of decision trees, most previous studies have difficulty ensuring the statistical optimality of a prediction of new data and suffer from overfitting because trees are usually used only to represent prediction functions to be…
The success of reinforcement learning in typical settings is predicated on Markovian assumptions on the reward signal by which an agent learns optimal policies. In recent years, the use of reward machines has relaxed this assumption by…
The reduced dynamics of a quantum system interacting with a linear heat bath finds an exact representation in terms of a stochastic Schr{\"o}dinger equation. All memory effects of the reservoir are transformed into noise correlations and…
Many optimization problems arising in high-dimensional statistics decompose naturally into a sum of several terms, where the individual terms are relatively simple but the composite objective function can only be optimized with iterative…
The systematic development of Coarse-Grained (CG) models via the Mori-Zwanzig projector operator formalism requires the explicit description of several terms, including a deterministic drift term, a dissipative memory term and a random…
Several fundamental problems that arise in optimization and computer science can be cast as follows: Given vectors $v_1,\ldots,v_m \in \mathbb{R}^d$ and a constraint family ${\cal B}\subseteq 2^{[m]}$, find a set $S \in \cal{B}$ that…
We present an algorithm for the simulation of the exact real-time dynamics of classical many-body systems with discrete energy levels. In the same spirit of kinetic Monte Carlo methods, a stochastic solution of the master equation is found,…
We establish numerical methods for solving the martingale optimal transport problem (MOT) - a version of the classical optimal transport with an additional martingale constraint on transport's dynamics. We prove that the MOT value can be…
The linear response of non-equilibrium systems with Markovian dynamics satisfies a generalized fluctuation-dissipation relation derived from time symmetry and antisymmetry properties of the fluctuations. The relation involves the sum of two…
Functional lifting methods provide a tool for approximating solutions of difficult non-convex problems by embedding them into a larger space. In this work, we investigate a mathematically rigorous formulation based on embedding into the…
This paper is devoted to the theoretical and numerical investigation of an augmented Lagrangian method for the solution of optimization problems with geometric constraints. Specifically, we study situations where parts of the constraints…
Stochastic approximation is a powerful class of algorithms with celebrated success. However, a large body of previous analysis focuses on stochastic approximations driven by contractive operators, which is not applicable in some important…
We derive the general analytical expressions for the statistical uncertainties of cumulants up to fourth order including an efficiency correction. The analytical expressions have been tested with a toy Monte Carlo model analysis. An…
Recent advances in quantum technologies and related experiments have created a need for highly accurate, versatile, and computationally efficient simulation techniques for the dynamics of open quantum systems. Long-lived correlation effects…
The main objective of this paper is to develop a martingale-type solution to optimal consumption--investment choice problems ([Merton, 1969] and [Merton, 1971]) under time-varying incomplete preferences driven by externalities such as…
Atypically large fluctuations in macroscopic non-equilibrium systems continue to attract interest. Their probability can often be determined by the optimal fluctuation method (OFM). The OFM brings about a conditional variational problem,…
Dynamic Mode Decomposition (DMD) has emerged as a powerful tool for analyzing the dynamics of non-linear systems from experimental datasets. Recently, several attempts have extended DMD to the context of low-rank approximations. This…
Multiscale dynamics are ubiquitous in applications of modern science. Because of time scale separation between relatively small set of slowly evolving variables and (typically) much larger set of rapidly changing variables, direct numerical…
This paper develops online algorithms to track solutions of time-varying constrained optimization problems. Particularly, resembling workhorse Kalman filtering-based approaches for dynamical systems, the proposed methods involve…