Related papers: Phase transitions in optimal unsupervised learning

Machine learning has been successfully used to study phase transitions. One of the most popular approaches to identifying critical points from data without prior knowledge of the underlying phases is the learning-by-confusion scheme. As…

Brownian motion with known positive drift is sampled in stages until it crosses a positive boundary $a$. A family of multistage samplers that control the expected overshoot over the boundary by varying the stage size at each stage is shown…

Large language models based on the Transformer architecture have demonstrated impressive capabilities to learn in context. However, existing theoretical studies on how this phenomenon arises are limited to the dynamics of a single layer of…

Learning rate schedules are ubiquitously used to speed up and improve optimisation. Many different policies have been introduced on an empirical basis, and theoretical analyses have been developed for convex settings. However, in many…

A key element in transfer learning is representation learning; if representations can be developed that expose the relevant factors underlying the data, then new tasks and domains can be learned readily based on mappings of these salient…

Learning how to learn efficiently is a fundamental challenge for biological agents and a growing concern for artificial ones. To learn effectively, an agent must regulate its learning speed, balancing the benefits of rapid improvement…

Learning requires the traversal of inherently distinct cognitive states to produce behavioral adaptation. Yet, tools to explicitly measure these states with non-invasive imaging -- and to assess their dynamics during learning -- remain…

Learning performed over finite time is inherently irreversible. In Part~I of this series, we modeled learning as a transport process in the space of parameter distributions and derived the Epistemic Speed Limit (ESL), which lower-bounds…

We derive the optimal input states and the optimal quantum measurements for estimating the unitary action of a given symmetry group, showing how the optimal performance is obtained with a suitable use of entanglement. Optimality is defined…

We extend our study of phase transitions in the generalization behaviour of multilayer perceptrons with non-overlapping receptive fields to the problem of the influence of noise, concerning e.g. the input units and/or the couplings between…

Learning is a fundamental characteristic of living systems, enabling them to comprehend their environments and make informed decisions. These decision-making processes are inherently influenced by available information about their…

An asymmetric exclusion process with $N$ particles on $L$ sites is considered where particles can move one or two sites per infinitesimal time-step. An exact analysis for N=2 and a mean-field theory in comparison with simulations show…

We study a spectral initialization method that serves a key role in recent work on estimating signals in nonconvex settings. Previous analysis of this method focuses on the phase retrieval problem and provides only performance bounds. In…

In this paper with study phase transitions of the $q$-state Potts model, through a number of unsupervised machine learning techniques, namely Principal Component Analysis (PCA), $k$-means clustering, Uniform Manifold Approximation and…

Information divergence that measures the difference between two nonnegative matrices or tensors has found its use in a variety of machine learning problems. Examples are Nonnegative Matrix/Tensor Factorization, Stochastic Neighbor…

Inertia drift is a transitional maneuver between two sustained drift stages in opposite directions, which provides valuable insights for navigating consecutive sharp corners for autonomous racing.However, this can be a challenging scenario…

We develop the spectral edge analysis: phase transitions in neural network training -- grokking, capability gains, loss plateaus -- are controlled by the spectral gap of the rolling-window Gram matrix of parameter updates. In the extreme…

We analyze phase transitions in the conditional entropy of a sequence caused by a change in the conditional variables. Such transitions happen, for example, when training to learn the parameters of a system, since the transition from the…

Integrated task and motion planning problems describe a multi-modal state space, which is often abstracted as a set of smooth manifolds that are connected via sets of transitions states. One approach to solving such problems is to sample…

Phase estimation is known to be a robust method for single-qubit gate calibration in quantum computers, while Bayesian estimation is widely used in devising optimal methods for learning in quantum systems. We present Bayesian phase…