Related papers: Understanding scaling through history-dependent pr…
The word embedding space in neural models is skewed, and correcting this can improve task performance. We point out that most approaches for modeling, correcting, and measuring the symmetry of an embedding space implicitly assume that the…
The availability of large datasets requires an improved view on statistical laws in complex systems, such as Zipf's law of word frequencies, the Gutenberg-Richter law of earthquake magnitudes, or scale-free degree distribution in networks.…
Deep learning has been successfully applied to various tasks, but its underlying mechanism remains unclear. Neural networks associate similar inputs in the visible layer to the same state of hidden variables in deep layers. The fraction of…
The scaling properties of the roughness of surfaces grown by two different processes randomly alternating in time, are addressed. The duration of each application of the two primary processes is assumed to be independently drawn from given…
Recent theories suggest that Neural Scaling Laws arise whenever the task is linearly decomposed into power-law distributed units. Alternatively, scaling laws also emerge when data exhibit a hierarchically compositional structure, as is…
There are several mathematical models yielding 1/f noise. For example, 1/f spectrum can be obtained from stochastic sequence of pulses having power-law distribution of pulse durations or from nonlinear stochastic differential equations. We…
The nature of statistics, statistical mechanics and consequently the thermodynamics of stochastic systems is largely determined by how the number of states $W(N)$ depends on the size $N$ of the system. Here we propose a scaling expansion of…
Understanding the innovation process, that is the underlying mechanisms through which novelties emerge, diffuse and trigger further novelties is undoubtedly of fundamental importance in many areas (biology, linguistics, social science and…
Symbolic regression (SR) aims to discover the underlying mathematical expressions that explain observed data. This holds promise for both gaining scientific insight and for producing inherently interpretable and generalizable models for…
Causal processes can give rise to distinctive distributions in the linguistic variables that they affect. Consequently, a secure understanding of a variable's distribution can hold a key to understanding the forces that have causally shaped…
We study survival time statistics in a noisy sample space reducing (SSR) process. Our simulations suggest that both the mean and standard deviation scale as $\sim N/N^{\lambda}$, where $N$ is the system size and $\lambda$ is a tunable…
We consider a class of real numbers, a subset of irrational numbers and certain mathematical constants, for which the elements in the simple continued fraction appears to be random. As an illustrative example, one can consider $\pi = \{x_0,…
Background: Zipf's law and Heaps' law are two representatives of the scaling concepts, which play a significant role in the study of complexity science. The coexistence of the Zipf's law and the Heaps' law motivates different understandings…
Stochastic equations constitute a major ingredient in many branches of science, from physics to biology and engineering. Not surprisingly, they appear in many quantitative studies of complex systems. In particular, this type of equation is…
We describe a continuous-time modelling framework for biological population dynamics that accounts for demographic noise. In the spirit of the methodology used by statistical physicists, transitions between the states of the system are…
We show that scaling arguments are very useful to analyze the dynamics of periodically modulated noisy systems. Information about the behavior of the relevant quantities, such as the signal-to-noise ratio, upon variations of the noise…
Power law distributions characterise several natural and social phenomena. The Zipf law for cities is one of those. The study views the question of whether that global regularity is independent of different spatial distributions of cities.…
We extend a generic class of systems which have previously been shown to spontaneously develop scaling (power law) distributions of their elementary degrees of freedom. While the previous systems were linear and exploded exponentially for…
In the last years, researchers have realized the difficulties of fitting power-law distributions properly. These difficulties are higher in Zipf's systems, due to the discreteness of the variables and to the existence of two representations…
We address the role of multiplicative stochastic processes in modeling the occurrence of power-law city size distributions. As an explanation of the result of Zipf's rank analysis, Simon's model is presented in a mathematically elementary…