Related papers: Entropy Collapse: A Universal Failure Mode of Inte…
Collapse, or a gravitational-like phase transition is studied in a microcanonical ensemble of particles with an attractive $1/r^{\alpha}$ potential. A mean field continuous integral equation is used to determine a saddle-point density…
Recursive learning -- where models are trained on data generated by previous versions of themselves -- is increasingly common in large language models, autonomous agents, and self-supervised systems. However, standard performance metrics…
Phase transitions abound in nature and society, and, from species extinction to stock market collapse, their prediction is of widespread importance. In earlier work we showed that Global Transfer Entropy, a general measure of information…
As frontier AI models are deployed in high-stakes decision pipelines, their ability to maintain metacognitive stability (knowing what they do not know, detecting errors, seeking clarification) under adversarial pressure is a critical safety…
We introduce a novel notion of invariance feedback entropy to quantify the state information that is required by any controller that enforces a given subset of the state space to be invariant. We establish a number of elementary properties,…
Differentiable matching layers and residual connection paradigms, often implemented via entropy-regularized Optimal Transport (OT), serve as critical mechanisms in structural prediction and architectural scaling. However, recovering…
Learning performed over finite time is inherently irreversible. In Part~I of this series, we modeled learning as a transport process in the space of parameter distributions and derived the Epistemic Speed Limit (ESL), which lower-bounds…
In a system of interdependent networks, an initial failure of nodes invokes a cascade of iterative failures that may lead to a total collapse of the whole system in a form of an abrupt first order transition. When the fraction of initial…
Neural Collapse is a phenomenon where the last-layer representations of a well-trained neural network converge to a highly structured geometry. In this paper, we focus on its first (and most basic) property, known as NC1: the within-class…
Experimental systems with a first order phase transition will often exhibit hysteresis when out of equilibrium. If defects are present, the hysteresis loop can have different shapes: with small disorder the hysteresis loop has a macroscopic…
Continual learning aims to acquire new tasks while preserving performance on previously learned ones, but most methods struggle with catastrophic forgetting. Existing approaches typically treat all layers uniformly, often trading stability…
We study the critical behavior of the entropy production of the Ising model subject to a magnetic field that oscillates in time. The mean-field model displays a phase transition that can be either first or second-order, depending on the…
Over the past decades, research on two-dimensional melting has established that both first-order and continuous hexatic-liquid transitions can occur, influenced by various factors in the potential energy and system details. The fundamental…
Entropy and order parameter are two key concepts in phase transition theory. This paper proposes an unified method to both find order parameter and estimate entropy automatically with unsupervised learning. The contributions of this paper…
Machine learning models must continuously self-adjust themselves for novel data distribution in the open world. As the predominant principle, entropy minimization (EM) has been proven to be a simple yet effective cornerstone in existing…
First-order irreversible phase transitions (IPT's) between an active regime and an absorbing state are studied in two models by means of both simulations and mean-field stability analysis. Hysteresis around coexistence is the result of the…
We demonstrate that dual entropy expressions of the Tsallis type apply naturally to statistical-mechanical systems that experience an exceptional contraction of their configuration space. The entropic index $\alpha>1$ describes the…
Interdependent networks are more fragile under random attacks than simplex networks, because interlayer dependencies lead to cascading failures and finally to a sudden collapse. This is a hybrid phase transition (HPT), meaning that at the…
Deep classifier neural networks enter the terminal phase of training (TPT) when training error reaches zero and tend to exhibit intriguing Neural Collapse (NC) properties. Neural collapse essentially represents a state at which the…
The Hartree-Fock approximation to the many-fermion problem can break exact symmetries, and in some cases by changing a parameter in the interaction one can drive the Hartree-Fock minimum from a symmetry-breaking state to a…