Related papers: Discrete surface growth process as a synchronizati…
$L_0$-smoothness, which has been pivotal to advancing decentralized optimization theory, is often fairly restrictive for modern tasks like deep learning. The recent advent of relaxed $(L_0,L_1)$-smoothness condition enables improved…
Very often, when studying topological or dynamical properties of random scale-free networks, it is tacitly assumed that degree-degree correlations are not present. However, simple constraints, such as the absence of multiple edges and…
Complex networks across various fields are often considered to be scale free -- a statistical property usually solely characterized by a power-law distribution of the nodes' degree $k$. However, this characterization is incomplete. In…
Recent analyses of neural networks with shaped activations (i.e. the activation function is scaled as the network size grows) have led to scaling limits described by differential equations. However, these results do not a priori tell us…
This paper proposes a novel proximal-gradient algorithm for a decentralized optimization problem with a composite objective containing smooth and non-smooth terms. Specifically, the smooth and nonsmooth terms are dealt with by gradient and…
In this work we determine a process-level Large Deviation Principle (LDP) for a model of interacting particles indexed by a lattice $\mathbb{Z}^d$. The connections are random, sparse and unscaled, so that the system converges in the large…
We analyze the correlation properties of the Erdos-Renyi random graph (RG) and the Barabasi-Albert scale-free network (SF) under the attack and repair strategy with detrended fluctuation analysis (DFA). The maximum degree k_max,…
Various real-life networks of current interest are simultaneously scale-free and modular. Here we study analytically the average distance in a class of deterministically growing scale-free modular networks. By virtue of the recursive…
In a recursive way and by including a parameter, we introduce a family of deterministic scale-free networks. The resulting networks exhibit small-world effects. We calculate the exact results for the degree exponent, the clustering…
An extension of the totally asymmetric exclusion process, which incorporates a dynamically extending lattice is explored. Although originally inspired as a model for filamentous fungal growth, here the dynamically extending exclusion…
We study the consensus decentralized optimization problem where the objective function is the average of $n$ agents private non-convex cost functions; moreover, the agents can only communicate to their neighbors on a given network topology.…
We propose and study a model of scale-free growing networks that gives a degree distribution dominated by a power-law behavior with a model-dependent, hence tunable, exponent. The model represents a hybrid of the growing networks based on…
While much progress has been achieved over the last decades in neuro-inspired machine learning, there are still fundamental theoretical problems in gradient-based learning using combinations of neurons. These problems, such as saddle points…
To construct continuum stochastic growth equations for competitive nonequilibrium surface-growth processes of the type RD+X that mixes random deposition (RD) with a correlated-growth process X, we use a simplex decomposition of the height…
In this article, we proposed a susceptible-infected model with identical infectivity, in which, at every time step, each node can only contact a constant number of neighbors. We implemented this model on scale-free networks, and found that…
Many networks exhibit scale free behavior where their degree distribution obeys a power law for large vertex degrees. Models constructed to explain this phenomena have relied on preferential attachment where the networks grow by the…
We present a scaling theory for the long time behavior of spontaneous imbibition in porous media consisting of interconnected pores with a large length-to-width ratio. At pore junctions the meniscus propagation in one or more branches can…
In this paper, we theoretically prove that gradient descent can find a global minimum of non-convex optimization of all layers for nonlinear deep neural networks of sizes commonly encountered in practice. The theory developed in this paper…
We study the overparametrization bounds required for the global convergence of stochastic gradient descent algorithm for a class of one hidden layer feed-forward neural networks, considering most of the activation functions used in…
Many complex networks display strong heterogeneity in the degree (connectivity) distribution. Heterogeneity in the degree distribution often reduces the average distance between nodes but, paradoxically, may suppress synchronization in…