Related papers: Selective pruning and neuronal death generate heav…

We study a class of growth algorithms for directed graphs that are candidate models for the evolution of genetic regulatory networks. The algorithms involve partial duplication of nodes and their links, together with innovation of new…

There is growing consensus among neuroscientists that neural circuits critical for survival are the result of genomic decompression processes. We introduce SynaptoGen, a novel computational framework--member of the Connectome Models…

We analyze the degree distribution's cut-off in finite size scale-free networks. We show that the cut-off behavior with the number of vertices $N$ is ruled by the topological constraints induced by the connectivity structure of the network.…

Neural Network Pruning has been established as driving force in the exploration of memory and energy efficient solutions with high throughput both during training and at test time. In this paper, we introduce a novel criterion for model…

Current deep neural networks (DNNs) are overparameterized and use most of their neuronal connections during inference for each task. The human brain, however, developed specialized regions for different tasks and performs inference with a…

Understanding how network function constrains neural connectivity is a central challenge in neuroscience. An influential approach is to train neural networks with gradient descent on cognitive tasks and characterize the resulting…

We review the recent fast progress in statistical physics of evolving networks. Interest has focused mainly on the structural properties of random complex networks in communications, biology, social sciences and economics. A number of giant…

Many complex natural and physical systems exhibit patterns of interconnection that conform, approximately, to a network structure referred to as scale-free. Preferential attachment is one of many algorithms that have been introduced to…

Self-organization of robust and efficient networks is important for a future design of communication or transportation systems, because both characteristics are not coexisting in many real networks. As one of the candidates for the…

A key contributing factor to incredible success of deep neural networks has been the significant rise on massively parallel computing devices allowing researchers to greatly increase the size and depth of deep neural networks, leading to…

Neural network pruning is a popular technique used to reduce the inference costs of modern, potentially overparameterized, networks. Starting from a pre-trained network, the process is as follows: remove redundant parameters, retrain, and…

The success of convolutional neural networks (CNNs) in various applications is accompanied by a significant increase in computation and parameter storage costs. Recent efforts to reduce these overheads involve pruning and compressing the…

With the general trend of increasing Convolutional Neural Network (CNN) model sizes, model compression and acceleration techniques have become critical for the deployment of these models on edge devices. In this paper, we provide a…

In recent years, deep neural networks have known a wide success in various application domains. However, they require important computational and memory resources, which severely hinders their deployment, notably on mobile devices or for…

We study structural properties of growing networks where both addition and deletion of nodes are possible. Our model network evolves via two independent processes. With rate r, a node is added to the system and this node links to a randomly…

Conventional scaling of neural networks typically involves designing a base network and growing different dimensions like width, depth, etc. of the same by some predefined scaling factors. We introduce an automated scaling approach…

Neural Networks and Decision Trees: two popular techniques for supervised learning that are seemingly disconnected in their formulation and optimization method, have recently been combined in a single construct. The connection pivots on…

Understanding and shaping the behaviour of Large Language Models (LLMs) is increasingly important as applications become more powerful and more frequently adopted. This paper introduces a machine unlearning method specifically designed for…

Preferential attachment is often suggested to be the underlying mechanism of the growth of a network, largely due to that many real networks are, to a certain extent, scale-free. However, such attribution is usually made under debatable…

We present analytical results for the emerging structure of networks that evolve via a combination of growth (by node addition and random attachment) and contraction (by random node deletion). To this end we consider a network model in…