Related papers: Asymptotic properties of one-layer artificial neur…
In this paper, we introduce a new class of stochastic multilayer networks. A stochastic multilayer network is the aggregation of $M$ networks (one per layer) where each is a subgraph of a foundational network $G$. Each layer network is the…
Understanding the inductive bias and generalization properties of large overparametrized machine learning models requires to characterize the dynamics of the training algorithm. We study the learning dynamics of large two-layer neural…
Gradient-based optimization methods implemented on distributed computing architectures are increasingly used to tackle large-scale machine learning applications. A key bottleneck in such distributed systems is the high communication…
We study the asymptotic properties of distributed consensus algorithms over switching directed random networks. More specifically, we focus on consensus algorithms over independent and identically distributed, directed random graphs, where…
When artificial neural networks have demonstrated exceptional practical success in a variety of domains, investigations into their theoretical characteristics, such as their approximation power, statistical properties, and generalization…
We deal with a random graph model evolving in discrete time steps by duplicating and deleting the edges of randomly chosen vertices. We prove the existence of an a.s. asymptotic degree distribution, with streched exponential decay; more…
We study the emergence of sparse representations in neural networks. We show that in unsupervised models with regularization, the emergence of sparsity is the result of the input data samples being distributed along highly non-linear or…
Although sparse neural networks have been studied extensively, the focus has been primarily on accuracy. In this work, we focus instead on network structure, and analyze three popular algorithms. We first measure performance when structure…
This research establishes that many real-world networks exhibit bounded expansion, a strong notion of structural sparsity, and demonstrates that it can be leveraged to design efficient algorithms for network analysis. We analyze several…
We investigate the robustness of sparse artificial neural networks trained with adaptive topology. We focus on a simple yet effective architecture consisting of three sparse layers with 99% sparsity followed by a dense layer, applied to…
We provide a discussion of several recent results which, in certain scenarios, are able to overcome a barrier in distributed stochastic optimization for machine learning. Our focus is the so-called asymptotic network independence property,…
In this paper, we consider asymptotic behaviors of multiscale multivalued stochastic systems with small noises. First of all, for general, fully coupled systems for multivalued stochastic differential equations of slow and fast motions with…
Many supervised machine learning methods are naturally cast as optimization problems. For prediction models which are linear in their parameters, this often leads to convex problems for which many mathematical guarantees exist. Models which…
Models of complex networks often incorporate node-intrinsic properties abstracted as hidden variables. The probability of connections in the network is then a function of these variables. Real-world networks evolve over time, and many…
Stochastic network influences complicate graph filter design by producing uncertainty in network iteration matrix eigenvalues, the points at which the graph filter response is defined. While joint statistics for the eigenvalues typically…
We introduce an evolving network model in which a new node attaches to a randomly selected target node and also to each of its neighbors with probability $p$. The resulting network is sparse for $p<\frac{1}{2}$ and dense (average degree…
In the light of the recently proposed scenario of asymmetry-induced synchronization (AISync), in which dynamical uniformity and consensus in a distributed system would demand certain asymmetries in the underlying network, we investigate…
In this paper we study the one dimensional random geometric graph when the location of the nodes are independent and exponentially distributed. We derive exact results and the limit theorems for the connectivity and other properties…
How does the shape of a network change as its size increases? Although random graph models provide some expectations for such "scaling behaviors" in the structure of networks, relatively little is known about how empirical network structure…
Most artificial networks today rely on dense representations, whereas biological networks rely on sparse representations. In this paper we show how sparse representations can be more robust to noise and interference, as long as the…