Related papers: Entropy Collapse: A Universal Failure Mode of Inte…
The empirical emergence of neural collapse -- a surprising symmetry in the feature representations of the training data in the penultimate layer of deep neural networks -- has spurred a line of theoretical research aimed at its…
Phase transitions mark qualitative reorganizations of collective behavior, yet identifying their boundaries remains challenging whenever analytic solutions are absent and conventional simulations fail. Here we introduce learnability as a…
We investigate the first-order phase transitions of the $q$-state Potts models with $q = 5, 6, 7$, and $8$ on the two-dimensional square lattice, using Monte Carlo simulations. At the very weakly first-order transition of the $q=5$ system,…
This work reports deep-learning-unique first-order and second-order phase transitions, whose phenomenology closely follows that in statistical physics. In particular, we prove that the competition between prediction error and model…
The appearance of a convex dip in the microcanonical entropy of finite systems usually signals a first order transition. However, a convex dip also shows up in some systems with a continuous transition as for example in the Baxter-Wu model…
We analyze phase transitions in the conditional entropy of a sequence caused by a change in the conditional variables. Such transitions happen, for example, when training to learn the parameters of a system, since the transition from the…
The efficiency of Monte Carlo samplers is dictated not only by energetic effects, such as large barriers, but also by entropic effects that are due to the sheer volume that is sampled. The latter effects appear in the form of an entropic…
RL training of multi-turn LLM agents is inherently unstable, and reasoning quality directly determines task performance. Entropy is widely used to track reasoning stability. However, entropy only measures diversity within the same input,…
The asymptotic behaviour near phase transitions can be suitably characterized by the scaling of $\Delta s/Q^2$ with $\epsilon=1-T/T_c$, where $\Delta s$ is the excess entropy and $Q$ is the order parameter. As $\Delta s$ is obtained by…
Large reasoning models have demonstrated remarkable performance on complex reasoning tasks, yet the excessive length of their chain-of-thought outputs remains a major practical bottleneck due to high computation cost and poor deployability.…
Two significant consequences of quantum fluctuations are entanglement and criticality. Entangled states may not be critical but a critical state shows signatures of universality in entanglement. A surprising result found here is that the…
Large Reasoning Models (LRMs) often suffer from overthinking, generating unnecessarily long reasoning chains even for simple tasks. This leads to substantial computational overhead with limited performance gain, primarily due to redundant…
The deep connection between entropy and information is discussed in terms of both classical and quantum physics. The mechanism of information transfer between systems via entanglement is explored in the context of decoherence theory. The…
The applicability of the highly idealized secondary infall model to `realistic' initial conditions is investigated. The collapse of proto-halos seeded by $3\sigma$ density perturbations to an Einstein--de Sitter universe is studied here for…
Neural networks have dramatically increased our capacity to learn from large, high-dimensional datasets across innumerable disciplines. However, their decisions are not easily interpretable, their computational costs are high, and building…
Under a time-dependent perturbation it is common to calculate the transition probability in going from from one eigenstate to another eigenstate of a quantum system. In this work we study the transition in going from a \textit{linear…
Entanglement entropies have revealed, in the last years, to be a powerful tool to extract information about the physics of condensed-matter systems. In the first part of this thesis, we show how to extract essential details about the…
Neural collapse (NC) -- the convergence of penultimate-layer features to a simplex equiangular tight frame -- is well understood at equilibrium, but the dynamics governing its onset remain poorly characterised. We identify a simple and…
We study the scaling behavior of the R\'enyi entanglement entropy with smooth boundaries at the phase transition point of the two-dimensional $J-Q_3$ model. Using the recently developed scaling formula [Deng {\it et al.}, Phys. Rev. B…
Learning depends on changes in synaptic connections deep inside the brain. In multilayer networks, these changes are triggered by error signals fed back from the output, generally through a stepwise inversion of the feedforward processing…