Related papers: Bias-Variance Trade-off and Model Selection for Pr…

In a regression model, prediction is typically performed after model selection. The large variability in the model selection makes the prediction unstable. Thus, it is essential to reduce the variability in model selection and improve…

A key issue in statistics and machine learning is to automatically select the "right" model complexity, e.g., the number of neighbors to be averaged over in k nearest neighbor (kNN) regression or the polynomial degree in regression with…

Within the past two decades, Gaussian process regression has been increasingly used for modeling dynamical systems due to some beneficial properties such as the bias variance trade-off and the strong connection to Bayesian mathematics. As…

Regression models fitted to data can be assessed on their goodness of fit, though models with many parameters should be disfavored to prevent over-fitting. Statisticians' tools for this are little known to physical scientists. These include…

It is a common phenomenon that for high-dimensional and nonparametric statistical models, rate-optimal estimators balance squared bias and variance. Although this balancing is widely observed, little is known whether methods exist that…

Consider a problem of predicting a response variable using a set of covariates in a linear regression model. If it is \emph{a priori} known or suspected that a subset of the covariates do not significantly contribute to the overall fit of…

The model selection procedure is usually a single-criterion decision making in which we select the model that maximizes a specific metric in a specific set, such as the Validation set performance. We claim this is very naive and can perform…

We introduce a new principle for model selection in regression and classification. Many regression models are controlled by some smoothness or flexibility or complexity parameter c, e.g. the number of neighbors to be averaged over in k…

While the traditional formulation of machine learning tasks is in terms of performance on average, in practice we are often interested in how well a trained model performs on rare or difficult data points at test time. To achieve more…

The question of the determination of the proton charge radius $R_p$ from electron scattering data led to an unprecedented experimental effort for measurements of the electric form factor of the proton at low and very low momentum transfer…

Regression models are used in a wide range of applications providing a powerful scientific tool for researchers from different fields. Linear, or simple parametric, models are often not sufficient to describe complex relationships between…

Regression plays a key role in many research areas and its variable selection is a classic and major problem. This study emphasizes cost of predictors to be purchased for future use, when we select a subset of them. Its economic aspect is…

Background: The recent disagreement between the proton charge radius extracted from Lamb shift measurements of muonic and electronic hydrogen invites speculation that new physics may be to blame. Several proposals have been made for new…

Breakthroughs in machine learning are rapidly changing science and society, yet our fundamental understanding of this technology has lagged far behind. Indeed, one of the central tenets of the field, the bias-variance trade-off, appears to…

The problem of selecting the most useful features from a great many (eg, thousands) of candidates arises in many areas of modern sciences. An interesting problem from genomic research is that, from thousands of genes that are active…

Static properties of hadrons such as their radii and other moments of the electric and magnetic distributions can only be extracted using theoretical methods and not directly measured from experiments. As a result, discrepancies between the…

We introduce an algorithm which, in the context of nonlinear regression on vector-valued explanatory variables, chooses those combinations of vector components that provide best prediction. The algorithm devotes particular attention to…

Classical learning theory suggests that the optimal generalization performance of a machine learning model should occur at an intermediate model complexity, with simpler models exhibiting high bias and more complex models exhibiting high…

We consider a new criterion-based approach to model selection in linear regression. Properties of selection criteria based on p-values of a likelihood ratio statistic are studied for families of linear regression models. We prove that such…

We extract the proton charge radius from the elastic form factor (FF) data using a novel theoretical framework combining chiral effective field theory and dispersion analysis. Complex analyticity in the momentum transfer correlates the…