Related papers: Effective Degrees of Freedom: A Flawed Metaphor
Degrees of freedom is a fundamental concept in statistical modeling, as it provides a quantitative description of the amount of fitting performed by a given procedure. But, despite this fundamental role in statistics, its behavior not…
Common practice in modern machine learning involves fitting a large number of parameters relative to the number of observations. These overparameterized models can exhibit surprising generalization behavior, e.g., ``double descent'' in the…
Overparametrized interpolating models have drawn increasing attention from machine learning. Some recent studies suggest that regularized interpolating models can generalize well. This phenomenon seemingly contradicts the conventional…
Many complex systems satisfy a set of constraints on their degrees of freedom, and at the same time, they are able to work and adapt to different conditions. Here, we describe the emergence of this ability in a simplified model in which the…
The Guide to the Expression of Uncertainty in Measurement advocates the use of an 'effective number of degrees of freedom' for the calculation of an interval of measurement uncertainty. However, it does not describe how this number is to be…
Unlike conventional mechanisms, compliant mechanisms produce the desired deformations by exploiting elastic strain and do not need, therefore, moving parts. The number of degrees of freedom of a conventional mechanism, also called mobility,…
A central question in modern machine learning and imaging sciences is to quantify the number of effective parameters of vastly over-parameterized models. The degrees of freedom is a mathematically convenient way to define this number of…
Model selection in penalized regression critically depends on an accurate assessment of model complexity, commonly quantified through the effective degrees of freedom. While the Lasso admits a simple and unbiased characterization, given by…
Understanding degrees of freedom in classical mechanics is fundamental to characterizing physical systems. Counting them is usually easy, especially if we can assign them a clear meaning. However, the precise definition of a degree of…
Stapp claims that, when spatial degrees of freedom are taken into account, Everett quantum mechanics is ambiguous due to a "core basis problem." To examine an aspect of this claim I generalize the ideal measurement model to include…
Quantifying the complexity of feed-forward neural networks (FFNNs) remains challenging due to their nonlinear, hierarchical structure and numerous parameters. We apply generalized degrees of freedom (GDF) to measure model complexity in…
In this paper, we explore degrees of freedom in deep sigmoidal neural networks. We show that the degrees of freedom in these models is related to the expected optimism, which is the expected difference between test error and training error.…
We explore the relationship among model fidelity, experimental design, and parameter estimation in sloppy models. We show that the approximate nature of mathematical models poses challenges for experimental design in sloppy models. In many…
Open effective field theories provide a systematic framework for describing physical systems interacting with an environment whose microscopic details are unknown, unobservable, or uncalculable. A basic step in constructing any effective…
The description of complex physical phenomena often involves sophisticated models that rely on a large number of parameters, with many dimensions and scales. One practical way to simplify that kind of models is to discard some of the…
Parametricity states that polymorphic functions behave the same regardless of how they are instantiated. When developing polymorphic programs, Wadler's free theorems can serve as free specifications, which can turn otherwise partial…
This paper discusses a new measure that is adaptable to certain intervalic probability frameworks, possibility theory, and belief theory. As such, it has the potential for wide use in knowledge engineering, expert systems, and related…
We derive the degrees of freedom of the lasso fit, placing no assumptions on the predictor matrix $X$. Like the well-known result of Zou, Hastie and Tibshirani [Ann. Statist. 35 (2007) 2173-2192], which gives the degrees of freedom of the…
We use a formal correspondence between thermodynamics and inference, where the number of samples can be thought of as the inverse temperature, to study a quantity called ``learning capacity'' which is a measure of the effective…
A good process model is expected not only to reflect the behavior of the process, but also to be as easy to read and understand as possible. Because preferences vary across different applications, numerous measures provide ways to reflect…