Related papers: Contractivity of neural ODEs: an eigenvalue optimi…
In this note we consider SDEs of the type $\mathrm{d} X_t=[F (X_t) -A X_t] \mathrm{d} t +D \mathrm{d} W_t$ under the assumptions that $A$'s eigenvalues are all of positive real parts and $F (\cdot)$ has slower-than-linear growth rate. It is…
Under a smart grid paradigm, there has been an increase in sensor installations to enhance situational awareness. The measurements from these sensors can be leveraged for real-time monitoring, control, and protection. However, these…
The derivation of second-order ordinary differential equations (ODEs) as continuous-time limits of optimization algorithms has been shown to be an effective tool for the analysis of these algorithms. Additionally, discretizing…
The E(xplicit)I(implicit)N(null) method was developed recently to remove numerical instability from PDEs, adding and subtracting an operator $\mathcal{D}$ of arbitrary structure, treating the operator implicitly in one case, and explicitly…
Logic optimization constitutes a critical phase within the Electronic Design Automation (EDA) flow, essential for achieving desired circuit power, performance, and area (PPA) targets. These logic circuits are typically represented as…
Continuous deep learning models, referred to as Neural Ordinary Differential Equations (Neural ODEs), have received considerable attention over the last several years. Despite their burgeoning impact, there is a lack of formal analysis…
We consider the self-adjoint two-dimensional Schr\"odinger operator $H_\mu$ associated with the differential expression $-\Delta -\mu$ describing a particle exposed to an attractive interaction given by a measure $\mu$ supported in a closed…
We study differentially private (DP) algorithms for stochastic convex optimization: the problem of minimizing the population loss given i.i.d. samples from a distribution over convex loss functions. A recent work of Bassily et al. (2019)…
It was shown recently by Su et al. (2016) that Nesterov's accelerated gradient method for minimizing a smooth convex function $f$ can be thought of as the time discretization of a second-order ODE, and that $f(x(t))$ converges to its…
An algorithm is presented for momentum gradient descent optimization based on the first-order differential equation of the Newtonian dynamics. The fictitious mass is introduced to the dynamics of momentum for regularizing the adaptive…
We study the fundamental computational problem of approximating optimal transport (OT) equations using neural differential equations (Neural ODEs). More specifically, we develop a novel framework for approximating unbalanced optimal…
We introduce new algorithms and convergence guarantees for privacy-preserving non-convex Empirical Risk Minimization (ERM) on smooth $d$-dimensional objectives. We develop an improved sensitivity analysis of stochastic gradient descent on…
Most deep neural networks use simple, fixed activation functions, such as sigmoids or rectified linear units, regardless of domain or network structure. We introduce differential equation units (DEUs), an improvement to modern neural…
The vast majority of successful deep neural networks are trained using variants of stochastic gradient descent (SGD) algorithms. Recent attempts to improve SGD can be broadly categorized into two approaches: (1) adaptive learning rate…
We propose smoothed primal-dual algorithms for solving stochastic and smooth nonconvex optimization problems with linear inequality constraints. Our algorithms are single-loop and only require a single stochastic gradient based on one…
In this paper, we consider the computation of the degree of the Dieudonn\'e determinant of a linear symbolic matrix $A = A_0 + A_1 x_1 + \cdots + A_m x_m$, where each $A_i$ is an $n \times n$ polynomial matrix over $\mathbb{K}[t]$ and…
We introduce a new second-order inertial optimization method for machine learning called INNA. It exploits the geometry of the loss function while only requiring stochastic approximations of the function values and the generalized…
We consider optimization problems of the first eigenvalue of elliptic operators with applications to two-phase optimal design problems (also known as topology optimization problems) of conductivity and elasticity relaxed by homogenization.…
Estimation of Distribution Algorithms (EDAs) are stochastic heuristics that search for optimal solutions by learning and sampling from probabilistic models. Despite their popularity in real-world applications, there is little rigorous…
An algorithm for solving smooth nonconvex optimization problems is proposed that, in the worst-case, takes $\mathcal{O}(\epsilon^{-3/2})$ iterations to drive the norm of the gradient of the objective function below a prescribed positive…