Related papers: Getting in the Zone for Successful Scalability
Interpreting the representation and generalization powers has been a long-standing issue in the field of machine learning (ML) and artificial intelligence. This work contributes to uncovering the emergence of universal scaling laws in…
We present a limited empirical study of scaling laws for transfer learning in transformer models. More specifically, we examine a scaling law that incorporates a "transfer gap" term, indicating the effectiveness of pre-training on one…
The critical point of a topological phase transition is described by a conformal field theory, where finite-size corrections to energy are uniquely related to its central charge. We investigate the finite-size scaling away from criticality…
Sequential scaling is a prominent inference-time scaling paradigm, yet its performance improvements are typically modest and not well understood, largely due to the prevalence of heuristic, non-principled approaches that obscure clear…
We investigate strongly correlated non-Abelian plasmas out of equilibrium. Based on numerical simulations, we establish a self-similar scaling property for the time evolution of spatial Wilson loops that characterizes a universal state of…
Adaptive simulated annealing (ASA) is a global optimization algorithm based on an associated proof that the parameter space can be sampled much more efficiently than by using other previous simulated annealing algorithms. The author's ASA…
The Area Under Curve measure (AUC) seems apt to evaluate and compare diverse models, possibly without calibration. An important example of AUC application is the evaluation and benchmarking of models that predict faithfulness of generated…
The Unified Software Development Process (USDP) and UML have been now generally accepted as the standard methodology and modeling language for developing Object-Oriented Systems. Although Agent-based Systems introduces new issues, we…
Low-precision training is critical for optimizing the trade-off between model quality and training costs, necessitating the joint allocation of model size, dataset size, and numerical precision. While empirical scaling laws suggest that…
We consider the solvable neural scaling model with three parameters: data complexity, target complexity, and model-parameter-count. We use this neural scaling model to derive new predictions about the compute-limited, infinite-data scaling…
Model merging efficiently aggregates capabilities from multiple fine-tuned models into a single one, operating purely in parameter space without original data or expensive re-computation. Despite empirical successes, a unified theory for…
We describe a method for approximating the universal scaling functions for the Ising model in a field. By making use of parametric coordinates, the free energy scaling function has a polynomial series everywhere. Its form is taken to be a…
In this paper we investigate the universality and scaling properties of the well-known quantities in classical statistical mechanics near the quantum phase transition point. We show that transverse susceptibility and derivatives of…
Ensuring that classifiers are well-calibrated, i.e., their predictions align with observed frequencies, is a minimal and fundamental requirement for classifiers to be viewed as trustworthy. Existing methods for assessing multiclass…
Scaling law that rewards large datasets, complex models and enhanced data granularity has been observed in various fields of deep learning. Yet, studies on time series forecasting have cast doubt on scaling behaviors of deep learning…
The universal anomalous diffusion scaling is obtained for the semiclassical quantum Hall transition, which has been argued to describe samples with dissipation or correlated impurities. The results explain a discrepancy between existing…
This article provides a brief overview of the UML SP (UML Scientific Profile). It is an Object-Oriented Simulation Language and may find usage in OOS and ABS. UML SP allows for the application of Unified Process methodology for the…
We study models of interacting fermions in one dimension to investigate the crossover from integrability to non-integrability, i.e., quantum chaos, as a function of system size. Using exact diagonalization of finite-sized systems, we study…
We discuss the computational performance of the adaptive resolution technique in molecular simulation when it is compared with equivalent full coarse-grained and full atomistic simulations. We show that an estimate of its efficiency, within…
Finite-size scaling (FSS) for a critical phase transition ($t=0$) states that within a window of size $|t|\sim L^{-1/\nu}$, the scaling behavior of any observable $Q$ in a system of linear size $L$ asymptotically follows a scaling form as…