Related papers: Geometry and Generalization: Eigenvalues as predic…

We study the behavior of eigenvalues of matrix P_N + Q_N where P_N and Q_N are two N -by-N random orthogonal projections. We relate the joint eigenvalue distribution of this matrix to the Jacobi matrix ensemble and establish the universal…

Using our previously published algorithm, we analyze the eigenvectors of the generalized Laplacian for two metric graphs occurring in practical applications. As expected, localization of an eigenvector is rare and the network should be…

Typical amortized inference in variational autoencoders is specialized for a single probabilistic query. Here we propose an inference network architecture that generalizes to unseen probabilistic queries. Instead of an encoder-decoder pair,…

Modular, Jacobi, and mock-modular forms serve as generating functions for BPS black hole degeneracies. By training feed-forward neural networks on Fourier coefficients of automorphic forms derived from the Dedekind eta function, Eisenstein…

Recently, autoencoders (AEs) have gained interest for creating parametric and invertible projections of multidimensional data. Parametric projections make it possible to embed new, unseen samples without recalculating the entire projection,…

What do auto-encoders learn about the underlying data generating distribution? Recent work suggests that some auto-encoder variants do a good job of capturing the local manifold structure of data. This paper clarifies some of these previous…

We consider eigenvectors of the Hamiltonian $H_0$ perturbed by a generic perturbation $V$ modelled by a random matrix from the Gaussian Unitary Ensemble (GUE). Using the supersymmetry approach we derive analytical results for the statistics…

In an unsupervised attack on variational autoencoders (VAEs), an adversary finds a small perturbation in an input sample that significantly changes its latent space encoding, thereby compromising the reconstruction for a fixed decoder. A…

Autoencoders are a widespread tool in machine learning to transform high-dimensional data into a lowerdimensional representation which still exhibits the essential characteristics of the input. The encoder provides an embedding from the…

Square matrices represent linear self-maps of vector spaces, and their eigenpoints are the fixed points of the induced map on projective space. Likewise, polynomial self-maps of a projective space are represented by tensors. We study the…

We introduce a novel data-driven approach aimed at designing high-quality shape deformations based on a coarse localized input signal. Unlike previous data-driven methods that require a global shape encoding, we observe that…

We consider square matrices over $\mathbb{C}$ satisfying an identity relating their eigenvalues and the corresponding eigenvectors re-proved and discussed by Denton, Parker, Tao and Zhang, called the eigenvector-eigenvalue identity. We…

Grokking is the intriguing phenomenon where a model learns to generalize long after it has fit the training data. We show both analytically and numerically that grokking can surprisingly occur in linear networks performing linear tasks in a…

Deep neural networks can be trained in reciprocal space, by acting on the eigenvalues and eigenvectors of suitable transfer operators in direct space. Adjusting the eigenvalues, while freezing the eigenvectors, yields a substantial…

Autoencoders are neural network formulations where the input and output of the network are identical and the goal is to identify the hidden representation in the provided datasets. Generally, autoencoders project the data nonlinearly onto a…

The inductive biases of trained neural networks are difficult to understand and, consequently, to adapt to new settings. We study the inductive biases of linearizations of neural networks, which we show to be surprisingly good summaries of…

The eigenvalue correlations of random matrices from the Jacobi Unitary Ensemble have a known asymptotic behavior as their size tends to infinity. In the bulk of the spectrum the behavior is described in terms of the sine kernel, and at the…

Generalization, the ability to perform well beyond the training context, is a hallmark of biological and artificial intelligence, yet anticipating unseen failures remains a central challenge. Conventional approaches often take a…

Autoencoders exhibit impressive abilities to embed the data manifold into a low-dimensional latent space, making them a staple of representation learning methods. However, without explicit supervision, which is often unavailable, the…

We study the perturbative power-series expansions of the eigenvalues and eigenvectors of a general tridiagonal (Jacobi) matrix of dimension d. The(small) expansion parameters are being the entries of the two diagonals of length d-1…