Related papers: Entropy Collapse: A Universal Failure Mode of Inte…
One characterization of a chaotic system is the quick delocalization of quantum information (fast scrambling). One therefore expects that in such a system a state quickly becomes locally indistinguishable from its perturbations. In this…
We study the entropy production in a macroscopic nonequilibrium system that undergoes an order-disorder phase transition. Entropy production is a characteristic feature of nonequilibrium dynamics with broken detailed balance. It is found…
The entanglement entropy of a subsystem of a quantum system is expressed, in the replica approach, through analytic continuation with respect to n of the trace of the n-th power of the reduced density matrix. This trace can be thought of as…
This paper presents a neural network filter method based on contraction operators to address model collapse in recursive training of generative models. Unlike \cite{xu2024probabilistic}, which requires superlinear sample growth…
Although transformer-based models have shown exceptional empirical performance, the fundamental principles governing their training dynamics are inadequately characterized beyond configuration-specific studies. Inspired by empirical…
AI training datasets will inevitably contain AI-generated examples, leading to ``feedback'' in which the output of one model impacts the training of another. It is known that such iterative feedback can lead to model collapse, yet the…
We propose that unconstrained artificial intelligence obeys a Second Law analogous to thermodynamics, where ethical entropy, defined as a measure of divergence from intended goals, increases spontaneously without continuous alignment work.…
We find a series of topological phase transitions of increasing order, beyond the more standard second-order phase transition in a one-dimensional topological superconductor. The jumps in the order of the transitions depend on the range of…
Understanding nonequilibrium systems and the consequences of irreversibility for the system's behavior as compared to the equilibrium case, is a fundamental question in statistical physics. Here, we investigate two types of nonequilbrium…
We present a general framework for understanding and analyzing critical behaviour in gravitational collapse. We adopt the method of renormalization group, which has the following advantages. (1) It provides a natural explanation for various…
We theoretically analyze the Feedback Alignment (FA) algorithm, an efficient alternative to backpropagation for training neural networks. We provide convergence guarantees with rates for deep linear networks for both continuous and discrete…
Phase-transition phenomena in deep learning (grokking, emergent capabilities, and ontological reorganization under context shift) have been studied through several lenses, including representational compression, singular learning theory,…
The current paradigm of training deep neural networks for classification tasks includes minimizing the empirical risk that pushes the training loss value towards zero, even after the training error has been vanished. In this terminal phase…
In this article, we explore properties of pseudo entropy [1] in quantum field theories and spin systems from several approaches. Pseudo entropy is a generalization of entanglement entropy such that it depends on both an initial and final…
Bootstrap percolation is a wide class of monotone cellular automata with random initial state. In this work we develop tools for studying in full generality one of the three `universality' classes of bootstrap percolation models in two…
We investigate the informational aspect of (1+1)-dimensional directed percolation, a canonical model of a nonequilibrium continuous transition to a phase dominated by a single special state called the "absorbing" state. Using a tensor…
The entanglement entropy, ${\cal S}$, is an indicator of quantum correlations in the ground state of a many body quantum system. At a second-order quantum phase-transition point in one dimension ${\cal S}$ generally has a logarithmic…
Modern practice for training classification deepnets involves a Terminal Phase of Training (TPT), which begins at the epoch where training error first vanishes; During TPT, the training error stays effectively zero while training loss is…
The entanglement evolution after a quantum quench became one of the tools to distinguish integrable versus chaotic (non-integrable) quantum many-body dynamics. Following this line of thoughts, here we propose that the revivals in the…
We introduce and study numerically a directed two-dimensional sandpile automaton with probabilistic toppling (probability parameter p) which provides a good laboratory to study both self-organized criticality and the far-from-equilibrium…