Related papers: Combinatorial Geometry of Threshold-Linear Network…
The dynamics of complex networks, being a current hot topic of many scientific fields, is often coded through the corresponding Laplacian matrix. The spectrum of this matrix carries the main features of the networks' dynamics. Here we…
Thresholding--the pruning of nodes or edges based on their properties or weights--is an essential preprocessing tool for extracting interpretable structure from complex network data, yet existing methods face several key limitations.…
There are many surprising and perhaps counter-intuitive properties of optimization of deep neural networks. We propose and experimentally verify a unified phenomenological model of the loss landscape that incorporates many of them. High…
Linear networks provide valuable insights into the workings of neural networks in general. This paper identifies conditions under which the gradient flow provably trains a linear network, in spite of the non-strict saddle points present in…
Neural networks are a central technique in machine learning. Recent years have seen a wave of interest in applying neural networks to physical systems for which the governing dynamics are known and expressed through differential equations.…
We propose a hierarchically modular, dynamical neural network model whose architecture minimizes a specifically designed energy function and defines its temporal characteristics. The model has an internal and an external space that are…
The monotonic dependence of the outputs of a neural network on some of its inputs is a crucial inductive bias in many scenarios where domain knowledge dictates such behavior. This is especially important for interpretability and fairness…
Recurrent neural network (RNN)'s architecture is a key factor influencing its performance. We propose algorithms to optimize hidden sizes under running time constraint. We convert the discrete optimization into a subset selection problem.…
Complex dynamical systems are often modeled as networks, with nodes representing dynamical units which interact through the network's links. Gene regulatory networks, responsible for the production of proteins inside a cell, are an example…
The configuration space network (CSN) of a dynamical system is an effective approach to represent the ensemble of configurations sampled during a simulation and their dynamic connectivity. To elucidate the connection between the CSN…
To preserve previously learned representations, continual learning systems must strike a balance between plasticity, the ability to acquire new knowledge, and stability. This stability-plasticity dilemma affects how representations can be…
Across all scales of the physical world, dynamical systems can often be usefully represented as abstract networks that encode the system's units and inter-unit interactions. Understanding how physical rules shape the topological structure…
In this paper, we address the problem of dynamic network embedding, that is, representing the nodes of a dynamic network as evolving vectors within a low-dimensional space. While the field of static network embedding is wide and…
We present a selective review of statistical modeling of dynamic networks. We focus on models with latent variables, specifically, the latent space models and the latent class models (or stochastic blockmodels), which investigate both the…
As a granular material is compressed, the particles and forces within the system arrange to form complex heterogeneous structures. Force chains are a prime example and are thought to constrain bulk properties such as mechanical stability…
Bridging logical and algorithmic reasoning with modern machine learning techniques is a fundamental challenge with potentially transformative impact. On the algorithmic side, many NP-hard problems can be expressed as integer programs, in…
The collective dynamics of interacting dynamical units on a network crucially depends on the properties of the network structure. Rather than considering large but finite graphs to capture the network, one often resorts to graph limits and…
Many natural systems are organized as networks, in which the nodes interact in a time-dependent fashion. The object of our study is to relate connectivity to the temporal behavior of a network in which the nodes are (real or complex)…
From social interactions to the human brain, higher-order networks are key to describe the underlying network geometry and topology of many complex systems. While it is well known that network structure strongly affects its function, the…
Deep neural networks (DNNs) are powerful machine learning models and have succeeded in various artificial intelligence tasks. Although various architectures and modules for the DNNs have been proposed, selecting and designing the…