English
Related papers

Related papers: Covariance matrix completion via auxiliary informa…

200 papers

Computing the inverse covariance matrix (or precision matrix) of large data vectors is crucial in weak lensing (and multi-probe) analyses of the large scale structure of the universe. Analytically computed covariances are noise-free and…

Instrumentation and Methods for Astrophysics · Physics 2017-12-06 Oliver Friedrich , Tim Eifler

This paper develops a new framework, called modular regression, to utilize auxiliary information -- such as variables other than the original features or additional data sets -- in the training process of linear models. At a high level, our…

Methodology · Statistics 2023-11-27 Ying Jin , Dominik Rothenhäusler

Many machine learning algorithms require precise estimates of covariance matrices. The sample covariance matrix performs poorly in high-dimensional settings, which has stimulated the development of alternative methods, the majority based on…

Machine Learning · Statistics 2016-11-04 Daniel Bartz

We give a new framework for solving the fundamental problem of low-rank matrix completion, i.e., approximating a rank-$r$ matrix $\mathbf{M} \in \mathbb{R}^{m \times n}$ (where $m \ge n$) from random observations. First, we provide an…

Machine Learning · Computer Science 2023-08-08 Jonathan A. Kelner , Jerry Li , Allen Liu , Aaron Sidford , Kevin Tian

In high-stakes risk prediction, quantifying uncertainty through interval-valued predictions is essential for reliable decision-making. However, standard evaluation tools like the receiver operating characteristic (ROC) curve and the area…

Machine Learning · Computer Science 2026-02-05 Yuqi Li , Matthew M. Engelhard

We conduct a study of graphical models and discuss the quality of model selection approximation by formulating the problem as a detection problem and examining the area under the curve (AUC). We are specifically looking at the model…

Information Theory · Computer Science 2016-08-26 Navid Tafaghodi Khajavi , Anthony Kuh

The accurate computation of the covariance matrix of fitted model parameters is a somewhat neglected task in Statistics. Algorithms are given for computing accurate covariance matrices derived from computing the Hessian matrix by numerical…

Computation · Statistics 2021-05-12 Rose Baker

Complete randomization allows for consistent estimation of the average treatment effect based on the difference in means of the outcomes without strong modeling assumptions on the outcome-generating process. Appropriate use of the…

Methodology · Statistics 2021-08-03 Anqi Zhao , Peng Ding

Background: Medical decision-making impacts both individual and public health. Clinical scores are commonly used among a wide variety of decision-making models for determining the degree of disease deterioration at the bedside. AutoScore…

Assessing variability according to distinct factors in data is a fundamental technique of statistics. The method commonly regarded to as analysis of variance (ANOVA) is, however, typically confined to the case where all levels of a factor…

Methodology · Statistics 2013-03-15 Steven Geinitz , Reinhard Furrer

Many modern data analyses benefit from explicitly modeling dependence structure in data -- such as measurements across time or space, ordered words in a sentence, or genes in a genome. A gold standard evaluation technique is structured…

Machine Learning · Statistics 2020-12-02 Soumya Ghosh , William T. Stephenson , Tin D. Nguyen , Sameer K. Deshpande , Tamara Broderick

This paper studies the covariance matrix estimation for high-dimensional time series within a new framework that combines low-rank factor and latent variable-specific cluster structures. The popular methods based on assuming the sparse…

Methodology · Statistics 2025-02-25 Dong Li , Xinghao Qiao , Cheng Yu

There is a lack of simple and scalable algorithms for uncertainty quantification. Bayesian methods quantify uncertainty through posterior and predictive distributions, but it is difficult to rapidly estimate summaries of these…

Computation · Statistics 2016-12-28 Cheng Li , Sanvesh Srivastava , David B. Dunson

This paper studies the problem of recursively estimating the weighted adjacency matrix of a network out of a temporal sequence of binary-valued observations. The observation sequence is generated from nonlinear networked dynamics in which…

Systems and Control · Electrical Eng. & Systems 2019-12-06 Yu Xing , Xingkang He , Haitao Fang , Karl Henrik Johansson

Estimation of large covariance matrices has drawn considerable recent attention, and the theoretical focus so far has mainly been on developing a minimax theory over a fixed parameter space. In this paper, we consider adaptive covariance…

Statistics Theory · Mathematics 2012-11-05 T. Tony Cai , Ming Yuan

We consider the problem of learning error covariance matrices for robotic state estimation. The convergence of a state estimator to the correct belief over the robot state is dependent on the proper tuning of noise models. During inference,…

Robotics · Computer Science 2023-09-19 Mohamad Qadri , Zachary Manchester , Michael Kaess

Estimating covariance matrix from massive high-dimensional and distributed data is significant for various real-world applications. In this paper, we propose a data-aware weighted sampling based covariance matrix estimator, namely DACE,…

Machine Learning · Computer Science 2020-10-13 Xixian Chen , Haiqin Yang , Shenglin Zhao , Michael R. Lyu , Irwin King

Repeated measurements are common in many fields, where random variables are observed repeatedly across different subjects. Such data have an underlying hierarchical structure, and it is of interest to learn covariance/correlation at…

Methodology · Statistics 2023-06-13 Sunpeng Duan , Guo Yu , Juntao Duan , Yuedong Wang

Estimation of covariance matrices or their inverses plays a central role in many statistical methods. For these methods to work reliably, estimated matrices must not only be invertible but also well-conditioned. In this paper we present an…

Methodology · Statistics 2014-08-06 Eric C. Chi , Kenneth Lange

Network estimation and variable selection have been extensively studied in the statistical literature, but only recently have those two challenges been addressed simultaneously. In this paper, we seek to develop a novel method to…

Methodology · Statistics 2024-06-11 Nathan Osborne , Christine B. Peterson , Marina Vannucci