Related papers: State evolution beyond first-order methods I: Rigo…
Weighted finite-state machines are a fundamental building block of NLP systems. They have withstood the test of time -- from their early use in noisy channel models in the 1990s up to modern-day neurally parameterized conditional random…
We describe a novel constructive technique for devising efficient first-order methods for a wide range of large-scale convex minimization settings, including smooth, non-smooth, and strongly convex minimization. The technique builds upon a…
This paper establishes the existence and uniqueness of mild solutions to stationary Hamilton-Jacobi-Bellman (HJB) equations associated with infinite-horizon stochastic optimal control problems in separable Hilbert spaces. Our framework…
Predictive statistical mechanics is a form of inference from available data, without additional assumptions, for predicting reproducible phenomena. By applying it to systems with Hamiltonian dynamics, a problem of predicting the macroscopic…
Analyzing decision problems under uncertainty commonly relies on idealizing assumptions about the describability of the world, with the most prominent examples being the closed world and the small world assumption. Most assumptions are…
Recent analysis of classical algorithms resulted in their axiomatization as transition systems satisfying some simple postulates, and in the formulation of the Abstract State Machine Theorem, which assures us that any classical algorithm…
We propose a new class of filtering and smoothing methods for inference in high-dimensional, nonlinear, non-Gaussian, spatio-temporal state-space models. The main idea is to combine the ensemble Kalman filter and smoother, developed in the…
A non-statistical theory of continuous, but irreversible, evolution can be constructed in terms of the Cartan calculus. The fundamental postulate, for an evolutionary theory which admits irreversible processes, is that the topology of the…
We introduce an iterative method to search for time-optimal Hamiltonians that drive a quantum system between two arbitrary, and in general mixed, quantum states. The method is based on the idea of progressively improving the efficiency of…
Latent autoregressive processes are a popular choice to model time varying parameters. These models can be formulated as nonlinear state space models for which inference is not straightforward due to the high number of parameters. Therefore…
We extend a contraction mapping argument for ordinary state-dependent delay differential equations to evolutionary partial differential equations in the sense of R. Picard, that is, to equations of the form $\bigl(\partial_{t}…
Robust optimization (RO) is one of the key paradigms for solving optimization problems affected by uncertainty. Two principal approaches for RO, the robust counterpart method and the adversarial approach, potentially lead to excessively…
A central challenge to using first-order methods for optimizing nonconvex problems is the presence of saddle points. First-order methods often get stuck at saddle points, greatly deteriorating their performance. Typically, to escape from…
Evolution by natural selection can be seen an algorithm for generating creative solutions to difficult problems. More precisely, evolution by natural selection is a class of algorithms that share a set of properties. The question we address…
Algorithms for bilevel optimization often encounter Hessian computations, which are prohibitive in high dimensions. While recent works offer first-order methods for unconstrained bilevel problems, the constrained setting remains relatively…
We propose a new model for a measurement of a characteristic of a microscopic quantum state by a large system that selects stochastically the different eigenstates with appropriate quantum weights. Unlike previous works which formulate a…
Stochastic first-order methods are standard for training large-scale machine learning models. Random behavior may cause a particular run of an algorithm to result in a highly suboptimal objective value, whereas theoretical guarantees are…
Recently, the problem of obtaining a short regular expression equivalent to a given finite automaton has been intensively investigated. Algorithms for converting finite automata to regular expressions have an exponential blow-up in the…
This work proposes a framework, embedded within the Performance Estimation framework (PEP), for obtaining worst-case performance guarantees on stochastic first-order methods. Given a first-order method, a function class, and a noise model…
When a prediction algorithm serves a collection of users, disparities in prediction quality are likely to emerge. If users respond to accurate predictions by increasing engagement, inviting friends, or adopting trends, repeated learning…