Related papers: Finite size scaling in neural networks

Dense Associative Memories are high storage capacity variants of the Hopfield networks that are capable of storing a large number of memory patterns in the weights of the network of a given size. Their common formulations typically require…

Large information sizes in samples and features can be encoded to speed up the learning of statistical models based on linear algebra and remove unwanted signals. Encoding information can reduce both sample and feature dimension to a…

Inspired by biology's most sophisticated computer, the brain, neural networks constitute a profound reformulation of computational principles. Remarkably, analogous high-dimensional, highly-interconnected computational architectures also…

The finite-size scaling function and the leading corrections for the single species 1D coagulation model $(A + A \rightarrow A)$ and the annihilation model $(A + A \rightarrow \emptyset)$ are calculated. The scaling functions are universal…

The Little-Hopfield network is an auto-associative computational model of neural memory storage and retrieval. This model is known to robustly store collections of randomly generated binary patterns as stable-states of the network dynamics.…

Efforts to reduce the numerical precision of computations in deep learning training have yielded systems that aggressively quantize weights and activations, yet employ wide high-precision accumulators for partial sums in inner-product…

We study the storage capacity of quantum neural networks (QNNs) described as completely positive trace preserving (CPTP) maps, which act on an $N$-dimensional Hilbert space. We demonstrate that QNNs can store up to $N$ linearly independent…

We present analytical results for the finite-size scaling in d--dimensional O(N) systems with strong anisotropy where the critical exponents (e.g. \nu_{||} and \nu_{\perp}) depend on the direction. Prominent examples are systems with…

Compressing large neural networks is an important step for their deployment in resource-constrained computational platforms. In this context, vector quantization is an appealing framework that expresses multiple parameters using a single…

It has been hypothesized that some form of "modular" structure in artificial neural networks should be useful for learning, compositionality, and generalization. However, defining and quantifying modularity remains an open problem. We cast…

A microcanonical finite-size scaling ansatz is discussed. It exploits the existence of a well-defined transition point for systems of finite size in the microcanonical ensemble. The best data collapse obtained for small systems yields…

Accepting validity of self-consistent theory of localization by Vollhardt and Woelfle, we derive the finite-size scaling procedure used for studies of the critical behavior in d-dimensional case and based on the use of auxiliary quasi-1D…

We introduce the Neural Collaborative Subspace Clustering, a neural model that discovers clusters of data points drawn from a union of low-dimensional subspaces. In contrast to previous attempts, our model runs without the aid of spectral…

We introduce a new method for determining the critical indices of the deconfinement transition in gauge theories. The method is based on the finite size scaling behavior of the expectation value of simple lattice operators, such as the…

Modern deep learning-based recommendation systems exploit hundreds to thousands of different categorical features, each with millions of different categories ranging from clicks to posts. To respect the natural diversity within the…

We show that hyperscaling and finite-size scaling imply that the probability distribution of the order parameter in finite size critical systems exhibit data collapse. We consider the examples of equilibrium critical systems, and a…

We benchmark recently proposed tensor network based finite-size scaling analysis in Phys. Rev. B {\bf 107}, 205123 (2023) against two-dimensional classical 3-state clock model. Due to the higher complexity of the model, more complicated…

This article exposes the failure of some big neural networks to leverage added capacity to reduce underfitting. Past research suggest diminishing returns when increasing the size of neural networks. Our experiments on ImageNet LSVRC-2010…

We study the problem of determining the capacity of the binary perceptron for two variants of the problem where the corresponding constraint is symmetric. We call these variants the rectangle-binary-perceptron (RPB) and the…

The critical temperature of thin Fe layers on Ir(100) is measured through M\"o{\ss}bauer spectroscopy as a function of the layer thickness. From a phenomenological finite-size scaling analysis, we find an effective shift exponent lambda =…