Related papers: A probabilistic study of neural complexity

In many contexts, simpler models are preferable to more complex models and the control of this model complexity is the goal for many methods in machine learning such as regularization, hyperparameter tuning and architecture design. In deep…

We introduce evolving networks where new vertices preferentially connect to the more central parts of a network. This makes such networks compact. Finite networks grown under the preferential compactness mechanism have complex…

We derive upper bounds on the complexity of ReLU neural networks approximating the solution of a linear system given the matrix and the right-hand side. We focus on matrices which are symmetric positive definite and sparse, as they appear…

We define a class of probability distributions that we call simplicial mixture models, inspired by simplicial complexes from algebraic topology. The parameters of these distributions represent their topology and we show that it is possible…

We study the structural characteristics of complex networks using the representative eigenvectors of the adjacent matrix. The probability distribution function of the components of the representative eigenvectors are proposed to describe…

The complexity of a system, in general, makes it difficult to determine some or almost all matrix elements of its operators. The lack of accuracy acts as a source of randomness for the matrix elements which are also subjected to an external…

Small-world networks, i.e. networks displaying both a high clustering coefficient and a small characteristic path length, are obliquitous in nature. Since their identification, the "small-worldness" metric, as proposed by Humphries and…

In this work, we formally prove that, under certain conditions, if a neural network is invariant to a finite group then its weights recover the Fourier transform on that group. This provides a mathematical explanation for the emergence of…

As demonstrated in many areas of real-life applications, neural networks have the capability of dealing with high dimensional data. In the fields of optimal control and dynamical systems, the same capability was studied and verified in many…

Neural networks (NNs) are now routinely implemented on systems that must operate in uncertain environments, but the tools for formally analyzing how this uncertainty propagates to NN outputs are not yet commonplace. Computing tight bounds…

Neural networks achieve outstanding accuracy in classification and regression tasks. However, understanding their behavior still remains an open challenge that requires questions to be addressed on the robustness, explainability and…

We propose a complexity measure which addresses the functional flexibility of networks. It is conjectured that the functional flexibility is reflected in the topological diversity of the assigned graphs, resulting from a resolution of their…

Algorithmic modeling relies on limited information in data to extrapolate outcomes for unseen scenarios, often embedding an element of arbitrariness in its decisions. A perspective on this arbitrariness that has recently gained interest is…

Recent work on neural scaling laws demonstrates that model performance scales predictably with compute budget, model size, and dataset size. In this work, we develop scaling laws based on problem complexity. We analyze two fundamental…

When the information about uncertainty cannot be quantified in a simple, probabilistic way, the topic of possibilistic decision theory is often a natural one to consider. The development of possibilistic decision theory has lead to a series…

These notes introduce probabilistic landscape models defined on high-dimensional discrete sequence spaces. The models are motivated primarily by fitness landscapes in evolutionary biology, but links to statistical physics and computer…

We study the build up of complexity on the example of 1 kg matter in different forms. We start on the simplest example of ideal gases, and then continue with more complex chemical, biological, life and social and technical structures. We…

Understanding how network function constrains neural connectivity is a central challenge in neuroscience. An influential approach is to train neural networks with gradient descent on cognitive tasks and characterize the resulting…

Genetic regulatory networks are defined by their topology and by a multitude of continuously adjustable parameters. Here we present a class of simple models within which the relative importance of topology vs. interaction strengths becomes…

This paper settles the sample complexity of single-parameter revenue maximization by showing matching upper and lower bounds, up to a poly-logarithmic factor, for all families of value distributions that have been considered in the…