Regularization Using Synthetic Data in High-Dimensional Models

To address the challenges of reliable statistical inference in high-dimensional models, we introduce the Synthetic-data Regularized Estimator (SRE). Unlike traditional regularization methods, the SRE regularizes the complex target model via a weighted likelihood based on synthetic data generated from a simpler, more stable model. This method provides a theoretically sound and practically effective alternative to parameter penalization. We establish key theoretical properties of the SRE in generalized linear models, including existence, stability, consistency, and minimax rate optimality. Applying the Convex Gaussian Min-Max Theorem, we derive a precise asymptotic characterization in the high-dimensional linear regime. To deal with the non-separable regularization, we introduce a novel decomposition in our analysis. Building upon these results, we develop practical methodologies for tuning parameter selection, confidence interval construction, and calibrated variable selection in high-dimensional inference. The effectiveness of the SRE is demonstrated through simulation studies and real-data applications.