Related papers: Geometric and Spectral Alignment for Deep Neural N…

Geometric variations like rotation, scaling, and viewpoint changes pose a significant challenge to visual understanding. One common solution is to directly model certain intrinsic structures, e.g., using landmarks. However, it then becomes…

We prove analogues for Cartan geometries of Gromov's major theorems on automorphisms of rigid geometric structures. The starting point is a Frobenius theorem, which says that infinitesimal automorphisms of sufficiently high order integrate…

Neural networks (NNs) are central to modern machine learning and achieve state-of-the-art results in many applications. However, the relationship between loss geometry and generalization is still not well understood. The local geometry of…

Structured Low-Rank Approximation is a problem arising in a wide range of applications in Numerical Analysis and Engineering Sciences. Given an input matrix $M$, the goal is to compute a matrix $M'$ of given rank $r$ in a linear or affine…

Dual complex numbers can represent rigid body motion in 2D spaces. Dual complex matrices are linked with screw theory, and have potential applications in various areas. In this paper, we study low rank approximation of dual complex…

Recently, the theory of dense graph limits has received attention from multiple disciplines including graph theory, computer science, statistical physics, probability, statistics, and group theory. In this paper we initiate the study of the…

This paper studies a regularized matrix tri-factorization \(A\approx PDQ\), where \(P\) and \(Q\) are side factors and \(D\) is a central core whose conditioning can be explicitly regularized or constrained. The formulation is a structured…

Structured low-rank approximation is the problem of minimizing a weighted Frobenius distance to a given matrix among all matrices of fixed rank in a linear space of matrices. We study exact solutions to this problem by way of computational…

Learning expressive probabilistic models correctly describing the data is a ubiquitous problem in machine learning. A popular approach for solving it is mapping the observations into a representation space with a simple joint distribution,…

We show that the training problem of a deep linear neural network under the squared loss admits an exact convex reformulation in a lifted space over a generalized completely positive cone. The reformulation has the same optimal value as the…

If a locally cartesian closed category carries a weak factorisation system, then the left maps are stable under pullback along right maps if and only if the right maps are closed under pushforward along right maps. We refer to this…

Distances in a network capture relations between nodes and are the basis of centrality, similarity, and influence measures. Often, however, the relevance of a node $u$ to a node $v$ is more precisely measured not by the magnitude of the…

Bayesian network structure learning is the notoriously difficult problem of discovering a Bayesian network that optimally represents a given set of training data. In this paper we study the computational worst-case complexity of exact…

We study the problem of identity testing for depth-3 circuits of top fanin k and degree d. We give a new structure theorem for such identities. A direct application of our theorem improves the known deterministic d^{k^k}-time black-box…

We study approximation limits of single-hidden-layer neural networks with analytic activation functions under global coefficient constraints. Under uniform $\ell^1$ bounds, or more generally sub-exponential growth of the coefficients, we…

The main goal of this paper is to prove, in positive characteristic $p$, stability behavior for the graded Betti numbers in the periodic tails of the minimal resolutions of Frobenius powers of the homogeneous maximal ideals for very general…

This paper introduces a new parameterization of deep neural networks (both fully-connected and convolutional) with guaranteed $\ell^2$ Lipschitz bounds, i.e. limited sensitivity to input perturbations. The Lipschitz guarantees are…

We show that the representation cost of fully connected neural networks with homogeneous nonlinearities - which describes the implicit bias in function space of networks with $L_2$-regularization or with losses such as the cross-entropy -…

Methods and algorithms that work with data on nonlinear manifolds are collectively summarized under the term `Riemannian computing'. In practice, curvature can be a key limiting factor for the performance of Riemannian computing methods.…

In deep networks with small initialization, training exhibits long plateaus separated by sharp feature-acquisition transitions. Whereas shallow nonlinear networks and deep linear networks are well studied, extending these analyses to deep…