相关论文: Phase Transitions in the Output Distribution of La…

Large Language Models (LLMs) have demonstrated impressive performance. To understand their behaviors, we need to consider the fact that LLMs sometimes show qualitative changes. The natural world also presents such changes called phase…

Phase transitions have been proposed as the origin of emergent abilities in large language models (LLMs), where new capabilities appear abruptly once models surpass critical thresholds of scale. Prior work, such as that of Wei et al.,…

We analyze phase transitions in the conditional entropy of a sequence caused by a change in the conditional variables. Such transitions happen, for example, when training to learn the parameters of a system, since the transition from the…

Large language models (LLMs) exhibit unprecedentedly rich scaling behaviors. In physics, scaling behavior is closely related to phase transitions, critical phenomena, and field theory. To investigate the phase transition phenomena in LLMs,…

This paper explores the connection between dynamical system properties and statistical physics of ensembles of such systems. Simple models are used to give novel phase transitions; particularly for finite N particle systems with many…

A new formulation of statistical mechanics is put forward according to which a random variable characterizing a macroscopic body is postulated to be infinitely divisible. It leads to a parametric representation of partition function of an…

Critical transitions, or large changes in the state of a system after a small change in the system's external conditions or parameters, commonly occur in a wide variety of disciplines, from the biological and social sciences to physics.…

The Random Language Model, proposed as a simple model of human languages, is defined by the averaged model of a probabilistic context-free grammar. This grammar expresses the process of sentence generation as a tree graph with nodes having…

We study the time evolution of thermodynamic observables that characterise the dissipative nature of thermal relaxation after an instantaneous temperature quench. Combining tools from stochastic thermodynamics and large-deviation theory, we…

Is it possible to develop a `physics of language' which can explain the spatial, temporal and social patterns we see, and which can predict future change like we forecast the weather? Such a theory is likely to involve ideas from…

Methods and insights from statistical physics are finding an increasing variety of applications where one seeks to understand the emergent properties of a complex interacting system. One such area concerns the dynamics of language at a…

Change-point detection and estimation procedures have been widely developed in the literature. However, commonly used approaches in change-point analysis have mainly been focusing on detecting change-points within an entire time series…

We introduce a novel characterization of phase transitions based on hypothesis testing. In our formulation, a phase transition is defined as the breakdown of statistical indistinguishability under vanishing parameter perturbations in the…

In this paper, we consider the relationship between phase-type distributions and positive systems through practical examples. Phase-type distributions, commonly used in modelling dynamic systems, represent the temporal evolution of a set of…

A natural phenomenon occurring in a living system is an outcome of the dynamics of the specific biological network underlying the phenomenon. The collective dynamics have both deterministic and stochastic components. The stochastic nature…

Large language models (LLMs) often exhibit abrupt emergent behavior, whereby new abilities arise at certain points during their training. This phenomenon, commonly referred to as a ''phase transition'', remains poorly understood. In this…

Dynamical phase transitions are crucial features of the fluctuations of statistical systems, corresponding to boundaries between qualitatively different mechanisms of maintaining unlikely values of dynamical observables over long periods of…

Quantifying the speed of linguistic change is challenging due to the fact that the historical evolution of languages is sparsely documented. Consequently, traditional methods rely on phylogenetic reconstruction. In this paper, we propose a…

Change-point detection methods are proposed for the case of temporary failures, or transient changes, when an unexpected disorder is ultimately followed by a readjustment and return to the initial state. A base distribution of the…

If a given behavior of a multi-agent system restricts the phase variable to a invariant manifold, then we define a phase transition as change of physical characteristics such as speed, coordination, and structure. We define such a phase…