English
Related papers

Related papers: Enhanced Balancing of Bias-Variance Tradeoff in St…

200 papers

We explore an explicit link between stochastic gradient descent using common batching strategies and splitting methods for ordinary differential equations. From this perspective, we introduce a new minibatching strategy (called Symmetric…

Optimization and Control · Mathematics 2025-04-08 Luke Shaw , Peter A. Whalley

The ranking and selection problem is a popular framework in the simulation literature for studying optimal information collection. We study a version of this problem in which the simulation output for each design is normally distributed…

Optimization and Control · Mathematics 2025-09-03 Jianzhong Du , Ilya O. Ryzhov , Siyang Gao

In this paper, we develop a new accelerated stochastic gradient method for efficiently solving the convex regularized empirical risk minimization problem in mini-batch settings. The use of mini-batches is becoming a golden standard in the…

Optimization and Control · Mathematics 2017-09-20 Tomoya Murata , Taiji Suzuki

Shrinkage estimation usually reduces variance at the cost of bias. But when we care only about some parameters of a model, I show that we can reduce variance without incurring bias if we have additional information about the distribution of…

Statistics Theory · Mathematics 2017-11-01 Jann Spiess

Algorithmic stability is a central concept in statistics and learning theory that measures how sensitive an algorithm's output is to small changes in the training data. Stability plays a crucial role in understanding generalization,…

Statistics Theory · Mathematics 2026-01-21 Abhinav Chakraborty , Yuetian Luo , Rina Foygel Barber

Sampling noisy intermediate-scale quantum devices is a fundamental step that converts coherent quantum-circuit outputs to measurement data for running variational quantum algorithms that utilize gradient and Hessian methods in cost-function…

Quantum Physics · Physics 2023-07-04 Y. S. Teo

Covariate shift occurs when the distribution of input features differs between the training and testing phases. In covariate shift, estimating an unknown function's moment is a classical problem that remains under-explored, despite its…

Machine Learning · Statistics 2025-07-01 Zhen Zhang , Xin Liu , Shaoli Wang , Jiaye Teng

In the context of linear regression, we construct a data-driven convex loss function with respect to which empirical risk minimisation yields optimal asymptotic variance in the downstream estimation of the regression coefficients. At the…

Statistics Theory · Mathematics 2025-05-29 Oliver Y. Feng , Yu-Chun Kao , Min Xu , Richard J. Samworth

We lower bound the complexity of finding $\epsilon$-stationary points (with gradient norm at most $\epsilon$) using stochastic first-order methods. In a well-studied model where algorithms access smooth, potentially non-convex functions…

Optimization and Control · Mathematics 2022-03-01 Yossi Arjevani , Yair Carmon , John C. Duchi , Dylan J. Foster , Nathan Srebro , Blake Woodworth

In the era of Model-as-a-Service, organizations increasingly rely on third-party AI models for rapid deployment. However, the dynamic nature of emerging AI applications, the continual introduction of new datasets, and the growing number of…

Machine Learning · Computer Science 2026-02-10 Zihan Zhu , Yanqiu Wu , Qiongkai Xu

Asymptotic optimality is a key theoretical property in model averaging. Due to technical difficulties, existing studies rely on restricted weight sets or the assumption that there is no true model with fixed dimensions in the candidate set.…

Statistics Theory · Mathematics 2024-11-15 Wenchao Xu , Xinyu Zhang

Stochastic gradient optimization is the dominant learning paradigm for a variety of scenarios, from classical supervised learning to modern self-supervised learning. We consider stochastic gradient algorithms for learning problems whose…

Machine Learning · Statistics 2025-08-29 Facheng Yu , Ronak Mehta , Alex Luedtke , Zaid Harchaoui

Machine learning algorithms in high-dimensional settings are highly susceptible to the influence of even a small fraction of structured outliers, making robust optimization techniques essential. In particular, within the…

Machine Learning · Computer Science 2025-04-25 Changyu Gao , Andrew Lowy , Xingyu Zhou , Stephen J. Wright

Markov parameters play a key role in system identification. There exists many algorithms where these parameters are estimated using least-squares in a first, pre-processing, step, including subspace identification and multi-step…

Systems and Control · Electrical Eng. & Systems 2024-05-08 Jiabao He , Cristian R. Rojas , Håkan Hjalmarsson

We study parameter inference in simulation-based stochastic models where the analytical form of the likelihood is unknown. The main difficulty is that score evaluation as a ratio of noisy Monte Carlo estimators induces bias and instability,…

Machine Learning · Statistics 2025-10-31 Zehao Li , Zhouchen Lin , Yijie Peng

Stochastic processes have found numerous applications in science, as they are broadly used to model a variety of natural phenomena. Due to their intrinsic randomness and uncertainty, they are, however, difficult to characterize. Here, we…

We consider exact asymptotics of the minimax risk for global testing against sparse alternatives in the context of high dimensional linear regression. Our results characterize the leading order behavior of this minimax risk in several…

Statistics Theory · Mathematics 2020-03-03 Rajarshi Mukherjee , Subhabrata Sen

In this paper, we introduce an unbiased gradient simulation algorithms for solving convex optimization problem with stochastic function compositions. We show that the unbiased gradient generated from the algorithm has finite variance and…

Optimization and Control · Mathematics 2017-11-22 Jose Blanchet , Donald Goldfarb , Garud Iyengar , Fengpei Li , Chaoxu Zhou

Minimax lower bounds are pessimistic in nature: for any given estimator, minimax lower bounds yield the existence of a worst-case target vector $\beta^*_{worst}$ for which the prediction error of the given estimator is bounded from below.…

Statistics Theory · Mathematics 2017-10-10 Pierre C Bellec

While classical scaling, just like principal component analysis, is parameter-free, other methods for embedding multivariate data require the selection of one or several tuning parameters. This tuning can be difficult due to the…

Methodology · Statistics 2023-10-19 Ery Arias-Castro , Phong Alain Chau