Related papers: Geodesics in the Deep Linear Network
Geodesic problems involve computing trajectories between prescribed initial and final states to minimize a user-defined measure of distance, cost, or energy. They arise throughout physics and engineering -- for instance, in determining…
The geometry of weight spaces and functional manifolds of neural networks play an important role towards 'understanding' the intricacies of ML. In this paper, we attempt to solve certain open questions in ML, by viewing them through the…
Random geometric networks consist of 1) a set of nodes embedded randomly in a bounded domain $\mathcal{V} \subseteq \mathbb{R}^d$ and 2) links formed probabilistically according to a function of mutual Euclidean separation. We quantify how…
We use geometric invariant theory (GIT) to study the deep linear network (DLN). The Kempf-Ness theorem is used to establish that the $L^2$ regularizer is minimized on the balanced manifold. We introduce related balancing flows using the…
The Grassmann manifold of linear subspaces is important for the mathematical modelling of a multitude of applications, ranging from problems in machine learning, computer vision and image processing to low-rank matrix optimization problems,…
Most real-world networks are embedded in latent geometries. If a node in a network is found in the vicinity of another node in the latent geometry, the two nodes have a disproportionately high probability of being connected by a link. The…
We prove a generalized version of the Morse index theorem for geodesics endowed with a non positive definite metric tensor (semi-Riemannian manifolds). We apply the result to obtain lower estimates on the number of geodesics joining two…
In this article we classify totally geodesic submanifolds in arbitrary products of rank one symmetric spaces. Furthermore, we give infinitely many examples of irreducible totally geodesic submanifolds in Hermitian symmetric spaces with…
We introduce anti-invariant Riemannian submersions from cosymplectic manifolds onto Riemannian manifolds. We survey main results of anti-invariant Riemannian submersions defined on cosymplectic manifolds. We investigate necessary and…
We study geometric properties of the gradient flow for learning deep linear convolutional networks. For linear fully connected networks, it has been shown recently that the corresponding gradient flow on parameter space can be written as a…
We introduce a new family of deep neural network models. Instead of specifying a discrete sequence of hidden layers, we parameterize the derivative of the hidden state using a neural network. The output of the network is computed using a…
We show that certain right-invariant metrics endow the infinite-dimensional Lie group of all smooth orientation-preserving diffeomorphisms of the circle with a Riemannian structure. The study of the Riemannian exponential map allows us to…
In this paper, we study Riemannian zeroth-order optimization in settings where the underlying Riemannian metric $g$ is geodesically incomplete, and the goal is to approximate stationary points with respect to this incomplete metric. To…
Although Deep Learning (DL) has achieved success in complex Artificial Intelligence (AI) tasks, it suffers from various notorious problems (e.g., feature redundancy, and vanishing or exploding gradients), since updating parameters in…
We studied linear mappings in Beidleman near-vector spaces and explored their matrix representations using $R$-bases of $R$-subgroups. Additionally, we developed algorithms for determining the seed number and seed sets of $R$-subgroups…
We show that a wide class of geometrically defined overdetermined semilinear partial differential equations may be explicitly prolonged to obtain closed systems. As a consequence, in the case of linear equations we extract sharp bounds on…
We propose a geometrical approach to the investigation of Hamiltonian systems on (Pseudo) Riemannian manifolds. A new geometrical criterion of instability and chaos is proposed. This approach is more generic than well known reduction to the…
We propose practical deep Gaussian process models on Riemannian manifolds, similar in spirit to residual neural networks. With manifold-to-manifold hidden layers and an arbitrary last layer, they can model manifold- and scalar-valued…
Deep learning is the mainstream technique for many machine learning tasks, including image recognition, machine translation, speech recognition, and so on. It has outperformed conventional methods in various fields and achieved great…
We develop a new method for visualizing and refining the invariances of learned representations. Specifically, we test for a general form of invariance, linearization, in which the action of a transformation is confined to a low-dimensional…