Related papers: Symmetry Breaking in Neural Network Optimization: …
The extraordinary computational power of the brain may be related in part to the fact that each of the smaller neural networks that compose it can behave transiently in many different ways, depending on its inputs. Mathematically, input…
Recently, many works have demonstrated that Symmetric Non-negative Matrix Factorization~(SymNMF) enjoys a great superiority for various clustering tasks. Although the state-of-the-art algorithms for SymNMF perform well on synthetic data,…
Neural networks are becoming increasingly prevalent in software, and it is therefore important to be able to verify their behavior. Because verifying the correctness of neural networks is extremely challenging, it is common to focus on the…
A simple two-species asymmetric exclusion model in one dimension with bulk and boundary exchanges of particles is investigated for the existence of spontaneous symmetry breaking. The model is a generalization of the bridge model for which…
Artificial and biological agents cannon learn given completely random and unstructured data. The structure of data is encoded in the metric relationships between data points. In the context of neural networks, neuronal activity within a…
A novel neural network (NN) approach is proposed for constrained optimization. The proposed method uses a specially designed NN architecture and training/optimization procedure called Neural Optimization Machine (NOM). The objective…
Disobeying the classical wisdom of statistical learning theory, modern deep neural networks generalize well even though they typically contain millions of parameters. Recently, it has been shown that the trajectories of iterative…
Many real networks have been found to have a rich degree of symmetry, which is a very important structural property of complex network, yet has been rarely studied so far. And where does symmetry comes from has not been explained. To…
Supersymmetry is one of the most plausible and theoretically motivated frameworks for extending the Standard Model. However, any supersymmetry in Nature must be a broken symmetry. Dynamical supersymmetry breaking (DSB) is an attractive idea…
The growing energy and performance costs of deep learning have driven the community to reduce the size of neural networks by selectively pruning components. Similarly to their biological counterparts, sparse networks generalize just as…
The computational capabilities of a neural network are widely assumed to be determined by its static architecture. Here we challenge this view by establishing that a fixed neural structure can operate in fundamentally different…
This work provides an additional step in the theoretical understanding of neural networks. We consider neural networks with one hidden layer and show that when learning symmetric functions, one can choose initial conditions so that standard…
In order to develop complex relationships between their inputs and outputs, deep neural networks train and adjust large number of parameters. To make these networks work at high accuracy, vast amounts of data are needed. Sometimes, however,…
Symmetry and dominance breaking can be crucial for solving hard combinatorial search and optimisation problems, but the correctness of these techniques sometimes relies on subtle arguments. For this reason, it is desirable to produce…
We develop an entropy based framework for analyzing symmetry breaking in dynamical systems. Information transfer, which measures the directional exchange of entropy between observables, provides a quantitative early indicator of symmetry…
If supersymmetric particles are discovered, an important problem will be to determine how supersymmetry has been broken. At collider energies, supersymmetry breaking can be parameterised by soft supersymmetry breaking parameters. Several…
We present a general holistic theory for the organization of complex networks, both human-engineered and naturally-evolved. Introducing concepts of value of interactions and satisfaction as generic network performance measures, we show that…
On a variety of tasks, the performance of neural networks predictably improves with training time, dataset size and model size across many orders of magnitude. This phenomenon is known as a neural scaling law. Of fundamental importance is…
Symmetry breaking--the phenomenon in which the symmetry of a system is not inherited by its stable states--underlies pattern formation, superconductivity, and numerous other effects. Recent theoretical work has established the possibility…
Who is more important in a network? Who controls the flow between the nodes or whose contribution is significant for connections? Centrality metrics play an important role while answering these questions. The betweenness metric is useful…