Related papers: The asymptotic effect of tuning parameters

We study generalized additive partial linear models, proposing the use of polynomial spline smoothing for estimation of nonparametric functions, and deriving quasi-likelihood based estimators for the linear parameters. We establish…

Control algorithms such as model predictive control (MPC) and state estimators rely on a number of different parameters. The performance of the closed loop usually depends on the correct setting of these parameters. Tuning is often done…

One of the most surprising and exciting discoveries in supervised learning was the benefit of overparameterization (i.e. training a very large model) to improving the optimization landscape of a problem, with minimal effect on statistical…

We study a high-dimensional generalized linear model and penalized empirical risk minimization with $\ell_1$ penalty. Our aim is to provide a non-trivial illustration that non-asymptotic bounds for the estimator can be obtained without…

For deep learning practitioners, hyperparameter tuning for optimizing model performance can be a computationally expensive task. Though visualization can help practitioners relate hyperparameter settings to overall model performance,…

An important class of structural models studies the determinants of skill formation and the optimal timing of interventions. In this paper, I provide new identification results for these models and investigate the effects of seemingly…

In a statistical analysis in Particle Physics, nuisance parameters can be introduced to take into account various types of systematic uncertainties. The best estimate of such a parameter is often modeled as a Gaussian distributed variable…

In this paper, we consider asymptotics of the optimal value and the optimal solutions of parametric minimax estimation problems. Specifically, we consider estimators of the optimal value and the optimal solutions in a sample minimax problem…

Unlike its intercept, a linear classifier's weight vector cannot be tuned by a simple grid search. Hence, this paper proposes weight vector tuning of a generic binary linear classifier through the parameterization of a decomposition of the…

In this article we study a class of parameters with the so-called `mixed bias property'. For parameters with this property, the bias of the semiparametric efficient one step estimator is equal to the mean of the product of the estimation…

We consider exact asymptotics of the minimax risk for global testing against sparse alternatives in the context of high dimensional linear regression. Our results characterize the leading order behavior of this minimax risk in several…

In this paper, we provide a mathematical framework for improving generalization in a class of learning problems which is related to point estimations for modeling of high-dimensional nonlinear functions. In particular, we consider a…

We analyze the prediction error of ridge regression in an asymptotic regime where the sample size and dimension go to infinity at a proportional rate. In particular, we consider the role played by the structure of the true regression…

We investigate statistical properties of the optimal value of the Sample Average Approximation of stochastic programs, continuing the study in Kr\"atschmer (2023). Central Limit Theorem type results are derived for the optimal value. As a…

We develop a multiscale scanning method to find anomalies in a $d$-dimensional random field in the presence of nuisance parameters. This covers the common situation that either the baseline-level or additional parameters such as the…

High-dimensional limit theorems have been shown useful to derive tuning rules for finding the optimal scaling in random-walk Metropolis algorithms. The assumptions under which weak convergence results are proved are however restrictive: the…

Model selection criteria are one of the most important tools in statistics. Proofs showing a model selection criterion is asymptotically optimal are tailored to the type of model (linear regression, quantile regression, penalized…

We consider the tuning parameter selection rules for nuclear norm regularized multivariate linear regression (NMLR) in high-dimensional setting. High-dimensional multivariate linear regression is widely used in statistics and machine…

This paper proposes various new analysis techniques for Bayes networks in which conditional probability tables (CPTs) may contain symbolic variables. The key idea is to exploit scalable and powerful techniques for synthesis problems in…

The Alternating Direction Method of Multipliers (ADMM) has gained significant attention across a broad spectrum of machine learning applications. Incorporating the over-relaxation technique shows potential for enhancing the convergence rate…