Related papers: Singular Fluctuation as Specific Heat in Bayesian …
Singular statistical models-including mixtures, matrix factorization, and neural networks-violate regular asymptotics due to parameter non-identifiability and degenerate Fisher geometry. Although singular learning theory characterizes…
A statistical model or a learning machine is called regular if the map taking a parameter to a probability distribution is one-to-one and if its Fisher information matrix is always positive definite. If otherwise, it is called singular. In…
In statistical learning, models are classified as regular or singular depending on whether the mapping from parameters to probability distributions is injective. Most models with hierarchical structures or latent variables are singular, for…
Watanabe's singular learning theory provides a framework for asymptotic analysis of Bayesian model selection for statistical models with singularities, where traditional statistical regularity assumptions fail. Learning coefficients, also…
Bayesian model selection commonly relies on Laplace approximation or the Bayesian Information Criterion (BIC), which assume that the effective model dimension equals the number of parameters. Singular learning theory replaces this…
The singularities prevalent in classical thermodynamics largely stem from the "postulate of equal a priori probabilities" neglecting the physical constraints imposed by computational complexity. This paper introduces Complexity Window…
Grokking, the abrupt transition from memorization to generalisation after extended training, suggests the presence of competing solution basins with distinct statistical properties. We study this phenomenon through the lens of Singular…
We introduce a novel Information Criterion (IC), termed Learning under Singularity (LS), designed to enhance the functionality of the Widely Applicable Bayes Information Criterion (WBIC) and the Singular Bayesian Information Criterion…
In this work, we advocate for the importance of singular learning theory (SLT) as it pertains to the theory and practice of variational inference in Bayesian neural networks (BNNs). To begin, using SLT, we lay to rest some of the confusion…
We report on a study of the superconducting order parameter thermodynamic fluctuations in $\mathrm{YBa}_2\mathrm{Cu}_3\mathrm{O}_{7 - \delta}$, $\mathrm{Bi}_{2}\mathrm{Sr}_{2}\mathrm{CaCu}_{2}\mathrm{O}_{8+ \delta}$ and…
In this work, we have studied simple models that can be solved analytically to illustrate various fluctuation theorems. These fluctuation theorems provide symmetries individually to the distributions of physical quantities like the…
Recent advances have clarified theoretical learning accuracy in Bayesian inference, revealing that the asymptotic behavior of metrics such as generalization loss and free energy, assessing predictive accuracy, is dictated by a rational…
Realistic phenomena can be described more appropriately using generalized canonical ensemble, with proper parameter sets involved. We have generalized the Einstein's theory for specific heat of solid in Tsallis statistics, where the…
The effect of long wavelength fluctuations on the Mode-Coupling-Theory (MCT) dynamical singularity at $T_c$ in the $\beta$ regime is studied by means of the standard field-theoretical procedure for a genuine second-order phase transition.…
The standard field-theoretical procedure to study the effect of long wavelength fluctuations on a genuine second-order phase transition is applied to the Mode-Coupling-Theory (MCT) dynamical singularity at $T_c$ in the $\beta$ regime.…
Order parameter fluctuations for the two dimensional Ising model in the region of the critical temperature are presented. A locus of temperatures T*(L) and of magnetic fields B*(L) are identified, for which the probability density function…
The marginal likelihood or evidence in Bayesian statistics contains an intrinsic penalty for larger model sizes and is a fundamental quantity in Bayesian model comparison. Over the past two decades, there has been steadily increasing…
Carrying out explicitly the computation in a paradigmatic model of non-interacting systems, the Gaussian Model, we show the existence of a singular point in the probability distribution $P(M)$ of an extensive variable $M$. Interpreting…
The puzzling situation where some thermodynamic quantities require a single-gap description, while others need a more complex gap scenario, is discussed. Our approach reveals that in some cases, the conclusions based on measurements of only…
The absence of a simple fluctuation-dissipation theorem is a major obstacle for studying systems that are not in thermodynamic equilibrium. We show that for a fluid in a non-equilibrium steady state characterized by a constant temperature…