Related papers: The cavity method for analysis of large-scale pena…

Methods for reasoning under uncertainty are a key building block of accurate and reliable machine learning systems. Bayesian methods provide a general framework to quantify uncertainty. However, because of model misspecification and the use…

This paper proposes a penalized composite likelihood method for model selection in colored graphical Gaussian models. The method provides a sparse and symmetry-constrained estimator of the precision matrix, and thus conducts model selection…

For statistical modeling wherein the data regime is unfavorable in terms of dimensionality relative to the sample size, finding hidden sparsity in the ground truth can be critical in formulating an accurate statistical model. The so-called…

We show by direct calculation that the replica and cavity methods are exactly equivalent for the spectrum of Erdos-Renyi random graph. We introduce a variational formulation based on the cavity method and use it to find approximate…

We propose a robust and efficient approach to the problem of compressive phase retrieval in which the goal is to reconstruct a sparse vector from the magnitude of a number of its linear measurements. The proposed framework relies on…

Sparse signal recovery from a small number of random measurements is a well known NP-hard to solve combinatorial optimization problem, with important applications in signal and image processing. The standard approach to the sparse signal…

The tuning parameter selection strategy for penalized estimation is crucial to identify a model that is both interpretable and predictive. However, popular strategies (e.g., minimizing average squared prediction error via cross-validation)…

We propose dimension reduction methods for sparse, high-dimensional multivariate response regression models. Both the number of responses and that of the predictors may exceed the sample size. Sometimes viewed as complementary, predictor…

Sparse recovery and subset selection are fundamental problems in varied communities, including signal processing, statistics and machine learning. Herein, we focus on an important greedy algorithm for these problems: Backward Stepwise…

The growing environmental footprint of artificial intelligence (AI), especially in terms of storage and computation, calls for more frugal and interpretable models. Sparse models (e.g., linear, neural networks) offer a promising solution by…

In this paper, we review state-of-the-art methods for feature selection in statistics with an application-oriented eye. Indeed, sparsity is a valuable property and the profusion of research on the topic might have provided little guidance…

Sparse reconstruction approaches using the re-weighted l1-penalty have been shown, both empirically and theoretically, to provide a significant improvement in recovering sparse signals in comparison to the l1-relaxation. However, numerical…

The spherical $p$-spin is a fundamental model for glassy physics, thanks to its analytic solution achievable via the replica method. Unfortunately the replica method has some drawbacks: it is very hard to apply to diluted models and the…

Estimating covariance parameters for multivariate spatial Gaussian random fields is computationally challenging, as the number of parameters grows rapidly with the number of variables, and likelihood evaluation requires operations of order…

Gaussian processes (GPs) provide a framework for Bayesian inference that can offer principled uncertainty estimates for a large range of problems. For example, if we consider regression problems with Gaussian likelihoods, a GP model enjoys…

Sparse regression has recently emerged as an attractive approach for discovering models of spatiotemporally complex dynamics directly from data. In many instances, such models are in the form of nonlinear partial differential equations…

Penalized likelihood models are widely used to simultaneously select variables and estimate model parameters. However, the existence of weak signals can lead to inaccurate variable selection, biased parameter estimation, and invalid…

Regression analysis is commonly conducted in survey sampling. However, existing methods fail when the relationships vary across different areas or domains. In this paper, we propose a unified framework to study the group-wise covariate…

Gaussian graphical modeling has been widely used to explore various network structures, such as gene regulatory networks and social networks. We often use a penalized maximum likelihood approach with the $L_1$ penalty for learning a…

We propose a sparse coefficient estimation and automated model selection procedure for autoregressive (AR) processes with heavy-tailed innovations based on penalized conditional maximum likelihood. Under mild moment conditions on the…