Related papers: Random Sequential Renormalization of Networks I: A…

Characterization of local minima draws much attention in theoretical studies of deep learning. In this study, we investigate the distribution of parameters in an over-parametrized finite neural network trained by ridge regularized empirical…

In this paper, we establish universal approximation theorems for neural networks applied to general nonlinear ill-posed operator equations. In addition to the approximation error, the measurement error is also taken into account in our…

Analytic phenomenological scaling is carried out for the random field Ising model in general dimensions using a bar geometry. Domain wall configurations and their decorated profiles and associated wandering and other exponents…

Recurrent neural networks are good at solving prediction problems. However, finding a network that suits a problem is quite hard because their performance is strongly affected by their architecture configuration. Automatic architecture…

Gradient regularization (GR) has been shown to improve the generalizability of trained models. While Natural Gradient Descent has been shown to accelerate optimization in the initial phase of training, little attention has been paid to how…

Plasticity Loss is an increasingly important phenomenon that refers to the empirical observation that as a neural network is continually trained on a sequence of changing tasks, its ability to adapt to a new task diminishes over time. We…

Many dynamical systems can be successfully analyzed by representing them as networks. Empirically measured networks and dynamic processes that take place in these situations show heterogeneous, non-Markovian, and intrinsically correlated…

Temporal data modelling techniques with neural networks are useful in many domain applications, including time-series forecasting and control engineering. This paper aims at developing a recurrent version of stochastic configuration…

Complex networks have acquired a great popularity in recent years, since the graph representation of many natural, social and technological systems is often very helpful to characterize and model their phenomenology. Additionally, the…

Random reshuffling techniques are prevalent in large-scale applications, such as training neural networks. While the convergence and acceleration effects of random reshuffling-type methods are fairly well understood in the smooth setting,…

The degree-degree correlation is crucial in understanding the structural properties of and dynamics occurring upon network, and is often measured by the assortativity coefficient $r$. In this paper, we first study this measure in detail and…

We present a detailed analysis of the class of regression decision tree algorithms which employ a regulized piecewise-linear node-splitting criterion and have regularized linear models at the leaves. From a theoretic standpoint, based on…

We propose a renormalization group treatment of stochastically growing networks. As an example, we study percolation on growing scale-free networks in the framework of a real-space renormalization group approach. As a result, we find that…

We present a renormalization group (RG) procedure which works naturally on a wide class of interacting one-dimension models based on perturbed (possibly strongly) continuum conformal and integrable models. This procedure integrates Kenneth…

This paper is a review dealing with the study of large size random recurrent neural networks. The connection weights are selected according to a probability law and it is possible to predict the network dynamics at a macroscopic scale using…

The structural re-parameterization (SRP) technique is a novel deep learning technique that achieves interconversion between different network architectures through equivalent parameter transformations. This technique enables the mitigation…

Random walk neural networks (RWNNs) have emerged as a promising approach for graph representation learning, leveraging recent advances in sequence models to process random walks. However, under realistic sampling constraints, RWNNs often…

We propose a new picture of the renormalization group (RG) approach in the presence of disorder, which considers the RG trajectories of each random sample (realization) separately instead of the usual renormalization of the averaged free…

Degree distribution, or equivalently called degree sequence, has been commonly used to be one of most significant measures for studying a large number of complex networks with which some well-known results have been obtained. By contrast,…

Regularization can mitigate the generalization gap between training and inference by introducing inductive bias. Existing works have already proposed various inductive biases from diverse perspectives. However, none of them explores…