English
Related papers

Related papers: Information-Theoretic Lower Bounds for Compressive…

200 papers

We consider the problem of estimating a sparse linear regression vector $\beta^*$ under a gaussian noise model, for the purpose of both prediction and model selection. We assume that prior knowledge is available on the sparsity pattern,…

Statistics Theory · Mathematics 2012-08-21 Karim Lounici , Massimiliano Pontil , Alexandre B. Tsybakov , Sara van de Geer

Measurement samples are often taken in various monitoring applications. To reduce the sensing cost, it is desirable to achieve better sensing quality while using fewer samples. Compressive Sensing (CS) technique finds its role when the…

Information Theory · Computer Science 2016-11-18 Ying Li , Kun Xie , Xin Wang

Dictionary learning is the problem of estimating the collection of atomic elements that provide a sparse representation of measured/collected signals or data. This paper finds fundamental limits on the sample complexity of estimating…

Information Theory · Computer Science 2018-03-06 Zahra Shakeri , Waheed U. Bajwa , Anand D. Sarwate

Each year, deep learning demonstrates new and improved empirical results with deeper and wider neural networks. Meanwhile, with existing theoretical frameworks, it is difficult to analyze networks deeper than two layers without resorting to…

Machine Learning · Computer Science 2023-03-28 Hong Jun Jeon , Yifan Zhu , Benjamin Van Roy

This paper investigates the problem of signal estimation from undersampled noisy sub-Gaussian measurements under the assumption of a cosparse model. Based on generalized notions of sparsity, we derive novel recovery guarantees for the…

Information Theory · Computer Science 2021-02-23 Martin Genzel , Gitta Kutyniok , Maximilian März

We study nonconvex zeroth-order optimization (ZOO) in a high-dimensional space $\mathbb R^d$ for functions with approximately $s$-sparse gradients. To reduce the dependence on the dimensionality $d$ in the query complexity, high-dimensional…

Machine Learning · Computer Science 2025-01-30 Ruizhong Qiu , Hanghang Tong

Optimal data aggregation aimed at maximizing IoT network lifetime by minimizing constrained on-board resource utilization continues to be a challenging task. The existing data aggregation methods have proven that compressed sensing is…

Signal Processing · Electrical Eng. & Systems 2018-06-14 Amarlingam M , Pradeep Kumar Mishra , P Rajalakshmi , Sumohana S. Channappayya , C. S. Sastry

First-principles based lattice models allow the modeling of ab initio thermodynamics of crystalline mixtures for applications such as the construction of phase diagrams and the identification of ground state atomic orderings. The recent…

We consider minimum variance estimation within the sparse linear Gaussian model (SLGM). A sparse vector is to be estimated from a linearly transformed version embedded in Gaussian noise. Our analysis is based on the theory of reproducing…

Information Theory · Computer Science 2013-04-16 Alexander Jung , Sebastian Schmutzhard , Franz Hlawatsch , Zvika Ben-Haim , Yonina C. Eldar

In this paper, we study the number of measurements required to recover a sparse signal in ${\mathbb C}^M$ with $L$ non-zero coefficients from compressed samples in the presence of noise. For a number of different recovery criteria, we prove…

Information Theory · Computer Science 2007-11-05 Mehmet Akçakaya , Vahid Tarokh

Compressive sensing is a methodology for the reconstruction of sparse or compressible signals using far fewer samples than required by the Nyquist criterion. However, many of the results in compressive sensing concern random sampling…

Information Theory · Computer Science 2013-06-11 Atul Divekar , Deanna Needell

This paper demonstrates the robustness of Lipschitz-regularized $\alpha$-divergences as objective functionals in generative modeling, showing they enable stable learning across a wide range of target distributions with minimal assumptions.…

Machine Learning · Statistics 2025-09-09 Ziyu Chen , Hyemin Gu , Markos A. Katsoulakis , Luc Rey-Bellet , Wei Zhu

Sparsity is a ubiquitous feature of many real world signals such as natural images and neural spiking activities. Conventional compressed sensing utilizes sparsity to recover low dimensional signal structures in high ambient dimensions…

Statistics Theory · Mathematics 2018-07-02 Abbas Kazemipour

In this letter, we present a unified result for the stable recovery bound of Lq(0 < q < 1) optimization model in compressed sensing, which is a constrained Lq minimization problem aware of the noise in a linear system. Specifically, without…

Optimization and Control · Mathematics 2016-09-07 Zhi-Long Dong , Xiaoqi Yang , Yu-Hong Dai

In this paper, we consider the problem of noiseless non-adaptive probabilistic group testing, in which the goal is high-probability recovery of the defective set. We show that in the case of $n$ items among which $k$ are defective, the…

Information Theory · Computer Science 2021-07-30 Wei Heng Bay , Eric Price , Jonathan Scarlett

We propose novel necessary and sufficient conditions for a sensing matrix to be "$s$-good" - to allow for exact $\ell_1$-recovery of sparse signals with $s$ nonzero entries when no measurement noise is present. Then we express the error…

Optimization and Control · Mathematics 2014-04-11 Anatoli Juditsky , Arkadii S. Nemirovski

For Gaussian sampling matrices, we provide bounds on the minimal number of measurements $m$ required to achieve robust weighted sparse recovery guarantees in terms of how well a given prior model for the sparsity support aligns with the…

Numerical Analysis · Mathematics 2016-05-04 Bubacarr Bah , Rachel Ward

We prove optimal sampling bounds achieving $(1\pm\varepsilon)$-relative error for a broad class of Lipschitz continuous classification loss functions under various regularization terms. This includes important functions such as logistic and…

Machine Learning · Computer Science 2026-05-25 Meysam Alishahi , Alexander Munteanu , Simon Omlor , Jeff M. Phillips

Sparse recovery can recover sparse signals from a set of underdetermined linear measurements. Motivated by the need to monitor large-scale networks from a limited number of measurements, this paper addresses the problem of recovering sparse…

Information Theory · Computer Science 2015-03-20 Meng Wang , Weiyu Xu , Enrique Mallada , Ao Tang

As shown in [Blumensath and Davies 2009, Baraniuk et al. 2010], signals whose wavelet coefficients exhibit a rooted tree structure can be recovered using specially-adapted compressed sensing algorithms from just n=O(k) measurements, where k…

Information Theory · Computer Science 2016-09-08 Coralia Cartis , Andrew Thompson