Related papers: Understanding Mode Connectivity via Parameter Spac…
In the setting of continual learning, a network is trained on a sequence of tasks, and suffers from catastrophic forgetting. To balance plasticity and stability of network in continual learning, in this paper, we propose a novel network…
We consider Convolutional Neural Networks (CNNs) with 2D structured features that are symmetric in the spatial dimensions. Such networks arise in modeling pairwise relationships for a sequential recommendation problem, as well as secondary…
We consider the optimization problem associated with fitting two-layer ReLU networks with respect to the squared loss, where labels are assumed to be generated by a target network. Focusing first on standard Gaussian inputs, we show that…
Connectivity (or equivalently, unweighted maximum flow) is an important measure in graph theory and combinatorial optimization. Given a graph $G$ with vertices $s$ and $t$, the connectivity $\lambda(s,t)$ from $s$ to $t$ is defined to be…
Topology and nonlinearity are deeply connected. However, whether topological effects can arise solely from the structure of nonlinear interaction terms, and the nature of the resulting topological phases, remain to large extent open…
Broadcast networks allow one to model networks of identical nodes communicating through message broadcasts. Their parameterized verification aims at proving a property holds for any number of nodes, under any communication topology, and on…
Understanding the dynamics of neural network parameters during training is one of the key challenges in building a theoretical foundation for deep learning. A central obstacle is that the motion of a network in high-dimensional parameter…
Neural Collapse refers to the curious phenomenon in the end of training of a neural network, where feature vectors and classification weights converge to a very simple geometrical arrangement (a simplex). While it has been observed…
Despite its importance for practical applications, not much is known about the optimal shape of a network that connects in an efficient way a set of points. This problem can be formulated in terms of a multiplex network with a fast layer…
The fundamental idea of embedding a network in a metric space is rooted in the principle of proximity preservation. Nodes are mapped into points of the space with pairwise distance that reflects their proximity in the network. Popular…
We deal with algorithmic techniques for minimal cost input-connectivity while maintaining controllability of linear systems. The input matrix is assumed to be constrained in the sense that the set of states that each input (if present) can…
A computational framework is presented for the sampling of the energy surface of magnetic systems via the systematic identification of first-order saddle points that determine connectivity of metastable states and define the mechanisms of…
Permutation of any two hidden units yields invariant properties in typical deep generative neural networks. This permutation symmetry plays an important role in understanding the computation performance of a broad class of neural networks…
Effective analysis in neuroscience benefits significantly from robust conceptual frameworks. Traditional metrics of interbrain synchrony in social neuroscience typically depend on fixed, correlation-based approaches, restricting their…
We present a didactic introduction to spectral Dynamic Causal Modelling (DCM), a Bayesian state-space modelling approach used to infer effective connectivity from non-invasive neuroimaging data. Spectral DCM is currently the most widely…
Quantum-correlated networks distribute quantum resources such as squeezed and entangled states. These states are central to modern quantum technology, including photonic quantum computing, quantum communications, non-destructive biological…
Symmetries found through automorphisms or graph fibrations provide important insights in network analysis. Symmetries identify clusters of robust synchronization in the network which improves the understanding of the functionality of…
In a sensor network, in practice, the communication among sensors is subject to:(1) errors or failures at random times; (3) costs; and(2) constraints since sensors and networks operate under scarce resources, such as power, data rate, or…
Modeling signals as linear combinations of atoms from a dictionary is ubiquitous in modern signal processing. In the finite-dimensional setting, whenever atoms depend nonlinearly upon unknown parameters, the signal model is said to be…
Although overparameterized models have achieved remarkable practical success, their theoretical properties, particularly their generalization behavior, remain incompletely understood. The well known double descents phenomenon suggests that…