Related papers: Emergence of Scale-Free Syntax Networks

Isolated complex networks have been studied deeply in the last decades due to the fact that many real systems can be modeled using these types of structures. However, it is well known that the behavior of a system not only depends on…

Complex networks have non-trivial characteristics and appear in many real-world systems. Such networks are vitally important, but their full underlying dynamics are not completely understood. Utilizing new data sources, however, can unveil…

Language exhibits structure at different scales, ranging from subwords to words, sentences, paragraphs, and documents. To what extent do deep models capture information at these scales, and can we force them to better capture structure…

We present a novel type of weighted scale-free network model, in which the weight grows independently of the attachment of new nodes. The evolution of this network is thus determined not only by the preferential attachment of new nodes to…

We analyse the so-called small-world network model (originally devised by Strogatz and Watts), treating it, among other things, as a case study of non-linear coupled difference or differential equations. We derive a system of evolution…

The body plan of the fruit fly is determined by the expression of just a handful of genes. We show that the spatial patterns of expression for several of these genes scale precisely with the size of the embryo. Concretely, discrete…

Recent probing studies reveal that large language models exhibit linear subspaces that separate true from false statements, yet the mechanism behind their emergence is unclear. We introduce a transparent, one-layer transformer toy model…

In the domain of unsupervised learning most work on speech has focused on discovering low-level constructs such as phoneme inventories or word-like units. In contrast, for written language, where there is a large body of work on…

In spite of its relevance to the origin of complex networks, the interplay between form and function and its role during network formation remains largely unexplored. While recent studies introduce dynamics by considering rewiring processes…

Recently, emergence of signaling conventions, among which language is a prime example, draws a considerable interdisciplinary interest ranging from game theory, to robotics to evolutionary linguistics. Such a wide spectrum of research is…

Scaling up language models has been shown to predictably improve performance and sample efficiency on a wide range of downstream tasks. This paper instead discusses an unpredictable phenomenon that we refer to as emergent abilities of large…

Languages are continuously undergoing changes, and the mechanisms that underlie these changes are still a matter of debate. In this work, we approach language evolution through the lens of causality in order to model not only how various…

Real-life networks often encounter vertex dysfunctions, which are usually followed by recoveries after appropriate maintenances. In this paper we present our research on a model of scale-free networks whose vertices are regularly removed…

Many networks in natural and human-made systems exhibit scale-free properties and are small worlds. Now we show that people's understanding of complex systems in their cognitive maps also follow a scale-free topology. People focus on a few…

Real-world growth processes and scalings have been broadly categorized into three growth regimes with distinctly different properties and driving forces. The first two are characterized by a positive and constant feedback between growth and…

We investigate the selective forces that promote the emergence of modularity in nature. We demonstrate the spontaneous emergence of modularity in a population of individuals that evolve in a changing environment. We show that the level of…

Wide neural networks with linear output layer have been shown to be near-linear, and to have near-constant neural tangent kernel (NTK), in a region containing the optimization path of gradient descent. These findings seem counter-intuitive…

We introduce a family of synthetic languages with hierarchical structure -- generated by a broadcast process on trees -- for which the role of context length and reasoning in autoregressive generation can be analyzed precisely. At the heart…

We study numerically a model of nonequilibrium networks where nodes and links are added at each time step with aging of nodes and connectivity- and age-dependent attachment of links. By varying the effects of age in the attachment…

We present a syntactic dependency treebank for naturalistic child and child-directed speech in English (MacWhinney, 2000). Our annotations largely followed the guidelines of the Universal Dependencies project (UD (Zeman et al., 2022)), with…