Related papers: Testing statistical laws in complex systems
Morphological inflection is a popular task in sub-word NLP with both practical and cognitive applications. For years now, state-of-the-art systems have reported high, but also highly variable, performance across data sets and languages. We…
Deep neural networks exhibit empirical neural scaling laws, with error decreasing as a power law with increasing model or data size, across a wide variety of architectures, tasks, and datasets. This universality suggests that scaling laws…
Taylor's law, also known as fluctuation scaling in physics and the power-law variance function in statistics, is an empirical pattern widely observed across fields including ecology, physics, finance, and epidemiology. It states that the…
The Gutenberg-Richter power law distribution of earthquake sizes is one of the most famous example illustrating self-similarity. It is well-known that the Gutenberg-Richter distribution has to be modified for large seismic moments, due to…
Probability forecasts of events are routinely used in climate predictions, in forecasting default probabilities on bank loans or in estimating the probability of a patient's positive response to treatment. Scoring rules have long been used…
A family of information theoretic models of communication was introduced more than a decade ago to explain the origins of Zipf's law for word frequencies. The family is a based on a combination of two information theoretic principles:…
Tokenization is a fundamental step in natural language processing (NLP) and other sequence modeling domains, where the choice of vocabulary size significantly impacts model performance. Despite its importance, selecting an optimal…
Large language models with a huge number of parameters, when trained on near internet-sized number of tokens, have been empirically shown to obey neural scaling laws: specifically, their performance behaves predictably as a power law in…
A new non-parametric statistic is introduced for the characterization of deviations from power laws. It is tested on the distribution of seismic energies given by the Gutenberg-Richter law. Based on the two first statistical log-moments, it…
In particle-based algorithms, the effect of binary collisions is commonly described in a statistical way, using Monte Carlo techniques. It is shown that, in the relativistic regime, stringent constraints should be considered on the sampling…
Background: Zipf's discovery that word frequency distributions obey a power law established parallels between biological and physical processes, and language, laying the groundwork for a complex systems perspective on human communication.…
Background properties in experimental particle physics are typically estimated using large data sets. However, different events can exhibit different features because of the quantum mechanical nature of the underlying physics processes.…
There is a considerable literature in case-control logistic regression on whether or not non-confounding covariates should be adjusted for. However, only limited and ad hoc theoretical results are available on this important topic. A…
A set of data with positive values follows a Pareto distribution if the log-log plot of value versus rank is approximately a straight line. A Pareto distribution satisfies Zipf's law if the log-log plot has a slope of -1. Since many types…
In this paper, we first study convergence rates in the law of large numbers for independent and identically distributed random variables. We obtain a strong $L^p$-convergence version and a strongly almost sure convergence version of the law…
For many standard models of random structure, first-order logic sentences exhibit a convergence phenomenon on random inputs. The most well-known example is for random graphs with constant edge probability, where the probabilities of…
Deep learning has recently revealed the existence of scaling laws, demonstrating that model performance follows predictable trends based on dataset and model sizes. Inspired by these findings and fascinating phenomena emerging in the…
In this paper the Zipf-Mandelbrot law is revisited in the context of linguistics. Despite its widespread popularity the Zipf--Mandelbrot law can only describe the statistical behaviour of a rather restricted fraction of the total number of…
We investigate symbolic sequences and in particular information carriers as e.g. books and DNA--strings. First the higher order Shannon entropies are calculated, a characteristic root law is detected. Then the algorithmic entropy is…
In this paper, we consider the empirical spectral distribution of the sample correlation matrix and investigate its asymptotic behavior under mild assumptions on the data's distribution, when dimension and sample size increase at the same…