Related papers: Entropy Collapse: A Universal Failure Mode of Inte…
Transferring knowledge from one neural network to another has been shown to be helpful for learning tasks with few training examples. Prevailing fine-tuning methods could potentially contaminate pre-trained features by comparably high…
As the variety of systems displaying scale invariant characteristics are matched only by their number, it is becoming increasingly important to understand their fundamental and universal elements. Much work has attempted to apply 2nd order…
Test-Time Adaptation (TTA) enhances model robustness to out-of-distribution (OOD) data by updating the model online during inference, yet existing methods lack theoretical insights into the fundamental causes of performance degradation…
The entanglement entropy (EE) of quantum systems is often used as a test of low-energy descriptions by conformal field theory (CFT). Here we point out that this is not a reliable indicator, as the EE often shows the same behavior even when…
Weak multiplex percolation generalizes percolation to multi-layer networks, represented as networks with a common set of nodes linked by multiple types (colors) of edges. We report a novel discontinuous phase transition in this problem.…
Recent years have witnessed the huge success of deep neural networks (DNNs) in various tasks of computer vision and text processing. Interestingly, these DNNs with massive number of parameters share similar structural properties on their…
The training algorithms for AI systems all introduce far-from-equilibrium dynamical processes, and understanding the irreversibility of these algorithms is a fundamental step towards understanding the learning dynamics of modern AI systems.…
The entanglement entropy of a subsystem $A$ of a quantum system is expressed, in the replica method, through analytic continuation with respect to n of the trace of the n-th power of the reduced density matrix $\tr\rho_A^n$. We study the…
When deploying artificial agents in real-world environments where they interact with humans, it is crucial that their behavior is aligned with the values, social norms or other requirements of that environment. However, many environments…
Via extensive Monte Carlo simulations along with systematic analyses of corrections to scaling, we estimate the order parameter critical exponent $\beta$ of absorbing phase transitions in systems with two symmetric absorbing states. The…
We observe a novel 'multiple-descent' phenomenon during the training process of LSTM, in which the test loss goes through long cycles of up and down trend multiple times after the model is overtrained. By carrying out asymptotic stability…
We study fidelity and fidelity susceptibility by addition of entanglement of entropy in the one-dimensional quantum compass model in a transverse magnetic field numerically. The whole four recognized gapped regions in the ground state phase…
Grokking - the delayed transition from memorisation to generalisation in neural networks - remains poorly understood. We study this phenomenon through the geometry of learned representations and identify a consistent empirical signature…
Standard Transformers have a fixed computational depth, fundamentally limiting their ability to generalize to tasks requiring variable-depth reasoning, such as multi-hop graph traversal or nested logic. We propose a depth-recurrent…
When instructed to underperform on multiple-choice evaluations, do language models engage with question content or fall back on positional shortcuts? We map the boundary between these regimes using a six-condition adversarial…
Uncovering and understanding universal dynamics in matter far from equilibrium remains a key challenge. In this work, we identify a so far unrecognized form of universal behavior that emerges after a sudden symmetry-breaking quench at…
Deploying reinforcement learning in safety critical domains, from autonomous vehicles to medical decision support, is constrained by failures arising when systems encounter unfamiliar conditions. We argue that the fundamental bottleneck is…
Reinforcement learning with verifiable rewards (RLVR) has emerged as a prominent paradigm for enhancing the reasoning capabilities of large language models (LLMs). However, the entropy of LLMs usually collapses during RLVR training, leading…
The deployment of machine learning in high-stakes services relies on ``human-in-the-loop'' architectures to mitigate algorithmic uncertainty. However, existing static policies fail to address a fundamental tension: algorithms suffer from…
We examine the temporal evolution of the modular entropy and capacity (in particular, the fluctuation of the entanglement entropy) for systems of time-dependent oscillators coupled by a (time-dependent) parameter. Such models, through the…