Related papers: Learning Unitary Operators with Help From u(n)

First-order optimization algorithms, often preferred for large problems, require the gradient of the differentiable terms in the objective function. These gradients often involve linear operators and their adjoints, which must be applied…

This paper is on improving the training of binary neural networks in which both activations and weights are binary. While prior methods for neural network binarization binarize each filter independently, we propose to instead parametrize…

Binary optimization, a representative subclass of discrete optimization, plays an important role in mathematical optimization and has various applications in computer vision and machine learning. Usually, binary optimization problems are…

Denote by $M_n$ the set of $n\times n$ complex matrices. Let $f: M_n \rightarrow [0,\infty)$ be a continuous map such that $f(\mu UAU^*)= f(A)$ for any complex unit $\mu$, $A \in M_n$ and unitary $U \in M_n$, $f(X)=0$ if and only if $X=0$…

Operator learning is a recently developed generalization of regression to mappings between functions. It promises to drastically reduce expensive numerical integration of PDEs to fast evaluations of mappings between functional states of a…

Vanishing (and exploding) gradients effect is a common problem for recurrent neural networks with nonlinear activation functions which use backpropagation method for calculation of derivatives. Deep feedforward neural networks with many…

In a previous paper, we have given an algebraic model to the set of intervals. Here, we apply this model in a linear frame. We define a notion of diagonalization of square matrices whose coefficients are intervals. But in this case, with…

Receiver Operating Characteristic (ROC) curves are useful for evaluation in binary classification and changepoint detection, but difficult to use for learning since the Area Under the Curve (AUC) is piecewise constant (gradient zero almost…

The process of un-reduction, a sort of reversal of reduction by the Lie group symmetries of a variational problem, is explored in the setting of field theories. This process is applied to the problem of curve matching in the plane, when the…

In the context of modern machine learning, models deployed in real-world scenarios often encounter diverse data shifts like covariate and semantic shifts, leading to challenges in both out-of-distribution (OOD) generalization and detection.…

Invariance and equivariance to geometrical transformations have proven to be very useful inductive biases when training (convolutional) neural network models, especially in the low-data regime. Much work has focused on the case where the…

A critical step in developing circuits for quantum simulation is to synthesize a desired unitary operator using the circuit building blocks. Studying unitaries and their generators from the Lie algebraic perspective has given rise to…

Combinatorial optimization problems are typically tackled by the branch-and-bound paradigm. We propose a new graph convolutional neural network model for learning branch-and-bound variable selection policies, which leverages the natural…

This is a tutorial paper on Recurrent Neural Network (RNN), Long Short-Term Memory Network (LSTM), and their variants. We start with a dynamical system and backpropagation through time for RNN. Then, we discuss the problems of gradient…

Employing equivariance in neural networks leads to greater parameter efficiency and improved generalization performance through the encoding of domain knowledge in the architecture; however, the majority of existing approaches require an a…

Deep neural networks have achieved impressive supervised classification performance in many tasks including image recognition, speech recognition, and sequence to sequence learning. However, this success has not been translated to…

To address the peculiarities of directed and/or signed graphs, new Laplacian operators have emerged. For instance, in the case of directionality, we encounter the magnetic operator, dilation (which is underexplored), operators based on…

We introduce a novel algorithm for estimating optimal parameters of linearized assignment flows for image labeling. An exact formula is derived for the parameter gradient of any loss function that is constrained by the linear system of ODEs…

This is an introductory note concerning the distribution vectors in a unitary representation of a Lie group. We discuss the definition of matrix coefficients associated with a pair of distributions and how one can compute them. Most of the…

Model Agnostic Meta Learning (MAML) is widely used to find a good initialization for a family of tasks. Despite its success, a critical challenge in MAML is to calculate the gradient w.r.t. the initialization of a long training trajectory…