Related papers: Geometric Approaches to Matrix Normalization and G…

Finding a new mathematical representations for graph, which allows direct comparison between different graph structures, is an open-ended research direction. Having such a representation is the first prerequisite for a variety of machine…

In 2003, van Dam and Haemers posed a fundamental question in spectral graph theory: does there exist a ``sensible'' matrix whose spectrum determines a random graph up to isomorphism? This paper introduces the class of {\em natural graph…

We study hierarchical properties of optimal transportation networks with biological background. The networks are obtained as minimizers of an energy functional which involves a metabolic cost term of a power-law form with exponent…

We analyse the normal quotient structure of several infinite families of finite connected edge-transitive, four-valent oriented graphs. These families were singled out by Marusic and others to illustrate various different internal…

The key to generalization is controlling the complexity of the network. However, there is no obvious control of complexity -- such as an explicit regularization term -- in the training of deep networks for classification. We will show that…

In this paper, we study the dynamics of gradient descent in learning neural networks for classification problems. Unlike in existing works, we consider the linearly non-separable case where the training data of different classes lie in…

As a first application of our renormalisation group approach to non-local matrix models [hep-th/0305066], we prove (super-)renormalisability of Euclidean two-dimensional noncommutative \phi^4-theory. It is widely believed that this model is…

Graph Neural Networks (GNNs) typically operate by message-passing, where the state of a node is updated based on the information received from its neighbours. Most message-passing models act as graph convolutions, where features are mixed…

It is well known that the matrix exponential of a non-normal matrix can exhibit transient growth even when all eigenvalues of the matrix have negative real part, and similarly for the powers of the matrix when all eigenvalues have magnitude…

Matrix completion has attracted much interest in the past decade in machine learning and computer vision. For low-rank promotion in matrix completion, the nuclear norm penalty is convenient due to its convexity but has a bias problem.…

We study the asymmetric matrix factorization problem under a natural nonconvex formulation with arbitrary overparametrization. The model-free setting is considered, with minimal assumption on the rank or singular values of the observed…

In this work, we propose to employ information-geometric tools to optimize a graph neural network architecture such as the graph convolutional networks. More specifically, we develop optimization algorithms for the graph-based…

We introduce the concept of regular quantum graphs and construct connected quantum graphs with discrete symmetries. The method is based on a decomposition of the quantum propagator in terms of permutation matrices which control the way…

For a graph $G$, the generalized adjacency matrix $A_\alpha(G)$ is the convex combination of the diagonal matrix $D(G)$ and the adjacency matrix $A(G)$ and is defined as $A_\alpha(G)=\alpha D(G)+(1-\alpha) A(G)$ for $0\leq \alpha \leq 1$.…

We systematize the approach to the investigation of deep neural network landscapes by basing it on the geometry of the space of implemented functions rather than the space of parameters. Grouping classifiers into equivalence classes, we…

Natural gradient descent has a remarkable property that in the small learning rate limit, it displays an invariance with respect to network reparameterizations, leading to robust training behavior even for highly covariant network…

The theory of training deep networks has become a central question of modern machine learning and has inspired many practical advancements. In particular, the gradient descent (GD) optimization algorithm has been extensively studied in…

A natural generalization of a regular (or equitable) partition of a graph, which makes sense also for non-regular graphs, is the so-called weight-regular partition, which gives to each vertex $u\in V$ a weight that equals the corresponding…

The aim of this paper is to provide a mathematical analysis of transformer architectures using a self-attention mechanism with layer normalization. In particular, observed patterns in such architectures resembling either clusters or uniform…

Optimization problems occurring in a wide variety of physical design problems, including but not limited to optical engineering, quantum control, structural engineering, involve minimization of a simple cost function of the state of the…