Related papers: Understanding scaling through history-dependent pr…
Event correlation between aftershocks in the coherent noise model is studied by making use of natural time, which has recently been introduced in complex time-series analysis. It is found that the aging phenomenon and the associated scaling…
The successor representation (SR) provides a powerful framework for decoupling predictive dynamics from rewards, enabling rapid generalisation across reward configurations. However, the classical SR is limited by its inherent policy…
Here we describe how some important scaling laws observed in the distribution of languages on Earth can emerge from a simple computer simulation. The proposed language dynamics includes processes of selective geographic colonization,…
Representation learning that leverages large-scale labelled datasets, is central to recent progress in machine learning. Access to task relevant labels at scale is often scarce or expensive, motivating the need to learn from unlabelled…
We analyze the effects of noise on the permutation entropy of dynamical systems. We take as numerical examples the logistic map and the R\"ossler system. Upon varying the noise strengthfaster, we find a transition from an…
A most debated topic of the last years is whether simple statistical physics models can explain collective features of social dynamics. A necessary step in this line of endeavour is to find regularities in data referring to large scale…
A family of information theoretic models of communication was introduced more than a decade ago to explain the origins of Zipf's law for word frequencies. The family is a based on a combination of two information theoretic principles:…
We consider increasingly complex models of matrix denoising and dictionary learning in the Bayes-optimal setting, in the challenging regime where the matrices to infer have a rank growing linearly with the system size. This is in contrast…
Neural scaling laws describe how the performance of deep neural networks scales with key factors such as training data size, model complexity, and training time, often following power-law behaviors over multiple orders of magnitude. Despite…
Some systems cannot be predicted by classical theories and it is required the development of combined deterministic and stochastic theories that make used of noise for dynamical prediction. Noise is not always an interfering signal which…
Many complex systems have been shown to share universal properties of organization, such as scale independence, modularity and self-similarity. We borrow tools from statistical physics in order to study structural preferential attachment…
In this paper we quantify the statistical properties and dynamics of the frequency of hashtag use on Twitter. Hashtags are special words used in social media to attract attention and to organize content. Looking at the collection of all…
In this contribution we consider stochastic growth models in the Kardar-Parisi-Zhang universality class in 1+1 dimension. We discuss the large time distribution and processes and their dependence on the class on initial condition. This…
Scaling law that rewards large datasets, complex models and enhanced data granularity has been observed in various fields of deep learning. Yet, studies on time series forecasting have cast doubt on scaling behaviors of deep learning…
Computer simulations of first-order relaxation processes show that the spatial configurations of the system acquire an invariant shape once the stationary regime is attained. Inspired by them we find that, in any first-order relaxation…
We propose a stochastic model of a fragmentation process, developed by taking into account fragment lifetime as a function of their size based on the Gibrat process. If lifetime is determined by a power function of fragment size, numerical…
Ranked set sampling (RSS) is a stratified sampling method that improves efficiency over simple random sampling (SRS) by utilizing auxiliary information for ranking and stratification. While balanced RSS (BRSS) assumes equal allocation…
In machine learning, the scaling law describes how the model performance improves with the model and data size scaling up. From a learning theory perspective, this class of results establishes upper and lower generalization bounds for a…
Soft random sampling (SRS) is a simple yet effective approach for efficient training of large-scale deep neural networks when dealing with massive data. SRS selects a subset uniformly at random with replacement from the full data set in…
A simple sparse coding mechanism appears in the sensory systems of several organisms: to a coarse approximation, an input $x \in \R^d$ is mapped to much higher dimension $m \gg d$ by a random linear transformation, and is then sparsified by…