Related papers: Effective Degrees of Freedom: A Flawed Metaphor

We develop a method to evaluate the generalized degrees of freedom (GDF), which is a key quantity of a model selection criterion, for linear regression with sparse regularization. Using the replica method, GDF is expressed by the variables…

Time series similarity measures are highly relevant in a wide range of emerging applications including training machine learning models, classification, and predictive modeling. Standard similarity measures for time series most often…

This theoretical work considers the following conundrum: linear response theory is successfully used by scientists in numerous fields, but mathematicians have shown that typical low-dimensional dynamical systems violate the theory's…

We construct a thermodynamic potential that can guide training of a generative model defined on a set of binary degrees of freedom. We argue that upon reduction in description, so as to make the generative model computationally-manageable,…

The bias-variance trade-off is a central concept in supervised learning. In classical statistics, increasing the complexity of a model (e.g., number of parameters) reduces bias but also increases variance. Until recently, it was commonly…

We present a statistical perspective on boosting. Special emphasis is given to estimating potentially complex parametric or nonparametric models, including generalized linear and additive models as well as regression models for survival…

Conformal prediction is a model-free machine learning method for constructing prediction regions at a guaranteed coverage probability level. However, a data scientist often faces three challenges in practice: (i) the determination of a…

Calibration is a frequently invoked concept when useful label probability estimates are required on top of classification accuracy. A calibrated model is a function whose values correctly reflect underlying label probabilities. Calibration…

It has been suggested that adversarial examples cause deep learning models to make incorrect predictions with high confidence. In this work, we take the opposite stance: an overly confident model is more likely to be vulnerable to…

It is shown that background fields of a topological character usually introduced as such in compactified string theories correspond to quantum degrees of freedom which parametrise the freedom in choosing a representation of the zero mode…

We present and study approximate notions of dimensional and margin complexity, which correspond to the minimal dimension or norm of an embedding required to approximate, rather then exactly represent, a given hypothesis class. We show that…

The approximation of smooth functions with a spectral basis typically leads to rapidly decaying coefficients where the rate of decay depends on the smoothness of the function and vice-versa. The optimal number of degrees of freedom in the…

We study the sample complexity of stochastic convex optimization when problem parameters, e.g., the distance to optimality, are unknown. We pursue two strategies. First, we develop a reliable model selection method that avoids overfitting…

Parametricity is a key metatheoretic property of type systems, which implies strong uniformity & modularity properties of the structure of types within systems possessing it. In recent years, various systems of dependent type theory have…

For the study of complex synthetic and biological molecular systems by computer simulations one is still restricted to simple model systems or to by far too small time scales. To overcome this problem multiscale techniques are being…

Many economic theory models incorporate finiteness assumptions that, while introduced for simplicity, play a real role in the analysis. We provide a principled framework for scaling results from such models by removing these finiteness…

The analysis of data sometimes requires fitting many free parameters in a theory to a large number of data points. Questions naturally arise about the compatibility of specific subsets of the data, such as those from a particular experiment…

We study two notions of expressiveness, which have appeared in abstraction theory for model checking, and find them incomparable in general. In particular, we show that according to the most widely used notion, the class of Kripke Modal…

Conformal prediction provides a distribution-free framework for uncertainty quantification via prediction sets with exact finite-sample coverage. In low dimensions these sets are easy to interpret, but in high-dimensional or structured…

Many random combinatorial objects have a component structure whose joint distribution is equal to that of a process of mutually independent random variables, conditioned on the value of a weighted sum of the variables. It is interesting to…