Related papers: The asymptotic effect of tuning parameters
This paper extends the literature on the theoretical properties of synthetic controls to the case of non-linear generative models, showing that the synthetic control estimator is generally biased in such settings. I derive a lower bound for…
Here we propose a novel searching scheme for a tuning parameter in high-dimensional penalized regression methods to address variable selection and modeling when sample sizes are limited compared to the data dimensions. Our method is…
We propose novel parameter estimation algorithms for a class of dynamical systems with nonlinear parametrization. The class is initially restricted to smooth monotonic functions with respect to a linear functional of the parameters. We show…
Asymptotic equivalence results for nonparametric regression experiments have always assumed that the variances of the observations are known. In practice, however the variance of each observation is generally considered to be an unknown…
Algorithms often have tunable parameters that impact performance metrics such as runtime and solution quality. For many algorithms used in practice, no parameter settings admit meaningful worst-case bounds, so the parameters are made…
Tuning parameters in supervised learning problems are often estimated by cross-validation. The minimum value of the cross-validation error can be biased downward as an estimate of the test error at that same value of the tuning parameter.…
Optimization software enables the solution of problems with millions of variables and associated parameters. These parameters are, however, often uncertain and represented with an analytical description of the parameter's distribution or…
Thomas' partial likelihood estimator of regression parameters is widely used in the analysis of nested case-control data with Cox's model. This paper proposes a new estimator of the regression parameters, which is consistent and…
A recent article on generalised linear mixed model asymptotics, Jiang et al. (2022), derived the rates of convergence for the asymptotic variances of maximum likelihood estimators. If $m$ denotes the number of groups and $n$ is the average…
This paper studies the parameter tuning problem of positive linear systems for optimizing their stability properties. We specifically show that, under certain regularity assumptions on the parametrization, the problem of finding the…
Learning processes are useful methodologies able to improve knowledge of real phenomena. These are often dependent on hyperparameters, variables set before the training process and regulating the learning procedure. Hyperparameters…
We derive asymptotic properties of penalized estimators for singular models for which identifiability may break and the true parameter values can lie on the boundary of the parameter space. Selection consistency of the estimators is also…
Many machine learning models have important structural tuning parameters that cannot be directly estimated from the data. The common tactic for setting these parameters is to use resampling methods, such as cross--validation or the…
We propose a technique for the design and analysis of adaptation algorithms in dynamical systems. The technique applies both to systems with conventional Lyapunov-stable target dynamics and to ones of which the desired dynamics around the…
Recent empirical and theoretical analyses of several commonly used prediction procedures reveal a peculiar risk behavior in high dimensions, referred to as double/multiple descent, in which the asymptotic risk is a non-monotonic function of…
This paper discusses a general framework for smoothing parameter estimation for models with regular likelihoods constructed in terms of unknown smooth functions of covariates. Gaussian random effects and parametric terms may also be…
We consider nonparametric testing in a non-asymptotic framework. Our statistical guarantees are exact in the sense that Type I and II errors are controlled for any finite sample size. Meanwhile, one proposed test is shown to achieve minimax…
The bias of an estimator is defined as the difference of its expected value from the parameter to be estimated, where the expectation is with respect to the model. Loosely speaking, small bias reflects the desire that if an experiment is…
Transfer learning refers to the promising idea of initializing model fits based on pre-training on other data. We particularly consider regression modeling settings where parameter estimates from previous data can be used as anchoring…
Machine learning applications frequently come with multiple diverse objectives and constraints that can change over time. Accordingly, trained models can be tuned with sets of hyper-parameters that affect their predictive behavior (e.g.,…