Related papers: Geodesics in the Deep Linear Network
Geodesic nets on Riemannian manifolds form a natural class of stationary objects generalizing geodesics. Yet almost nothing is known about their classification or general properties even when the ambient Riemannian manifold is the Euclidean…
We study the Riemannian geometry of the Deep Linear Network (DLN) as a foundation for a thermodynamic description of the learning process. The main tools are the use of group actions to analyze overparametrization and the use of Riemannian…
Learning representations on Grassmann manifolds is popular in quite a few visual recognition tasks. In order to enable deep learning on Grassmann manifolds, this paper proposes a deep network architecture by generalizing the Euclidean…
Deep generative models are tremendously successful in learning low-dimensional latent representations that well-describe the data. These representations, however, tend to much distort relationships between points, i.e. pairwise distances…
Non-Euclidean constraints are inherent in many kinds of data in computer vision and machine learning, typically as a result of specific invariance requirements that need to be respected during high-level inference. Often, these geometric…
This paper addresses the understanding and characterization of residual networks (ResNet), which are among the state-of-the-art deep learning architectures for a variety of supervised learning problems. We focus on the mapping component of…
Latent space geometry provides a rigorous and empirically valuable framework for interacting with the latent variables of deep generative models. This approach reinterprets Euclidean latent spaces as Riemannian through a pull-back metric,…
Deep generative models learn a mapping from a low dimensional latent space to a high-dimensional data space. Under certain regularity conditions, these models parameterize nonlinear manifolds in the data space. In this paper, we investigate…
This article provides an expository account of training dynamics in the Deep Linear Network (DLN) from the perspective of the geometric theory of dynamical systems. Rigorous results by several authors are unified into a thermodynamic…
Machine learning problems have an intrinsic geometric structure as central objects including a neural network's weight space and the loss function associated with a particular task can be viewed as encoding the intrinsic geometry of a given…
In this paper we investigate possible extensions of the idea of geodesic completeness in complex manifolds, following two directions: metrics are somewhere allowed not to be of maximum rank, or to have 'poles' somewhere else. Geodesics are…
Recent methods in geometric deep learning have introduced various neural networks to operate over data that lie on Riemannian manifolds. Such networks are often necessary to learn well over graphs with a hierarchical structure or to learn…
A geodesic orbit manifold is a complete Riemannian manifold all of whose geodesics are orbits of one-parameter groups of isometries. We give both a geometric and an algebraic characterization of geodesic orbit manifolds that are…
We propose extrinsic and intrinsic deep neural network architectures as general frameworks for deep learning on manifolds. Specifically, extrinsic deep neural networks (eDNNs) preserve geometric features on manifolds by utilizing an…
In this note, we show that sub-Riemannian manifolds can contain branching normal minimizing geodesics. This phenomenon occurs if and only if a normal geodesic has a discontinuity in its rank at a non-zero time, which in particular for a…
We consider the problem of training a deep orthogonal linear network, which consists of a product of orthogonal matrices, with no non-linearity in-between. We show that training the weights with Riemannian gradient descent is equivalent to…
One of the main challenges in modern deep learning is to understand why such over-parameterized models perform so well when trained on finite data. A way to analyze this generalization concept is through the properties of the associated…
We construct a linear system on a general curve in a totally geodesic subvariety of the moduli space of curves. As a consequence, we obtain rank bounds for totally geodesic subvarieties of dimension at least two. Furthermore, we classify…
We compute the length of geodesics on a Riemannian manifold by regular polynomial interpolation of the global solution of the eikonal equation related to the line element $ds^2=g_{ij}dx^idx^j$ of the manifold. Our algorithm approximates the…
New perspective form of equations for geodesic lines in Riemann Geometry was found. This method is based on the use of differential forms in differential equations as arguments of differentiation. At that, these forms do not have a…