English
Related papers

Related papers: Model free variable importance for high dimensiona…

200 papers

This paper introduces novel weighted conformal p-values and methods for model-free selective inference. The problem is as follows: given test units with covariates $X$ and missing responses $Y$, how do we select units for which the…

Methodology · Statistics 2023-09-27 Ying Jin , Emmanuel J. Candès

Reliability-oriented sensitivity analysis aims at combining both reliability and sensitivity analyses by quantifying the influence of each input variable of a numerical model on a quantity of interest related to its failure. In particular,…

Statistics Theory · Mathematics 2022-10-25 Julien Demange-Chryst , François Bachoc , Jérôme Morio

Over the past few years, the use of machine learning models has emerged as a generic and powerful means for prediction purposes. At the same time, there is a growing demand for interpretability of prediction models. To determine which…

Machine Learning · Computer Science 2023-01-13 Joris Pries , Guus Berkelmans , Sandjai Bhulai , Rob van der Mei

Variable selection in high-dimensional space characterizes many contemporary problems in scientific discovery and decision making. Many frequently-used techniques are based on independence screening; examples include correlation ranking…

Methodology · Statistics 2008-12-18 Jianqing Fan , Richard Samworth , Yichao Wu

There are a variety of settings where vague prior information may be available on the importance of predictors in high-dimensional regression settings. Examples include ordering on the variables offered by their empirical variances (which…

Methodology · Statistics 2022-05-20 Benjamin G. Stokell , Rajen D. Shah

Variable importance plays a pivotal role in interpretable machine learning as it helps measure the impact of factors on the output of the prediction model. Model agnostic methods based on the generation of "null" features via permutation…

Importance sampling, which involves sampling from a probability density function (PDF) proportional to the product of an importance weight function and a base PDF, is a powerful technique with applications in variance reduction, biased or…

Machine Learning · Computer Science 2025-02-10 Heasung Kim , Taekyun Lee , Hyeji Kim , Gustavo de Veciana

Independence screening is a powerful method for variable selection for `Big Data' when the number of variables is massive. Commonly used independence screening methods are based on marginal correlations or variations of it. In many…

Statistics Theory · Mathematics 2012-11-02 Emre Barut , Jianqing Fan , Anneleen Verhasselt

A vital stage in the mathematical modelling of real-world systems is to calibrate a model's parameters to observed data. Likelihood-free parameter inference methods, such as Approximate Bayesian Computation, build Monte Carlo samples of the…

Computation · Statistics 2021-12-23 Thomas P Prescott , Ruth E Baker

Shapley values are widely used for model-agnostic data valuation and feature attribution, yet they implicitly assume contributors are interchangeable. This can be problematic when contributors are dependent (e.g., reused/augmented data or…

Machine Learning · Computer Science 2026-02-11 Kiljae Lee , Ziqi Liu , Weijing Tang , Yuan Zhang

When applied to high-dimensional datasets, feature selection algorithms might still leave dozens of irrelevant variables in the dataset. Therefore, even after feature selection has been applied, classifiers must be prepared to the presence…

Machine Learning · Computer Science 2018-11-21 Danilo Vasconcellos Vargas , Hirotaka Takano , Junichi Murata

Importance sampling (IS) is a powerful Monte Carlo methodology for the approximation of intractable integrals, very often involving a target probability density function. The performance of IS heavily depends on the appropriate selection of…

Computation · Statistics 2023-06-22 Víctor Elvira , Emilie Chouzenoux , Ömer Deniz Akyildiz , Luca Martino

Likelihood-free Bayesian inference algorithms are popular methods for calibrating the parameters of complex, stochastic models, required when the likelihood of the observed data is intractable. These algorithms characteristically rely…

Computation · Statistics 2021-12-23 Thomas P Prescott , David J Warne , Ruth E Baker

Many existing approaches for estimating feature importance are problematic because they ignore or hide dependencies among features. A causal graph, which encodes the relationships among input variables, can aid in assigning feature…

Machine Learning · Computer Science 2021-03-01 Jiaxuan Wang , Jenna Wiens , Scott Lundberg

Importance sampling (IS) is a Monte Carlo technique that relies on weighted samples, simulated from a proposal distribution, to estimate intractable integrals. The quality of the estimators improves with the number of samples. However, for…

Computation · Statistics 2022-07-18 Medha Agarwal , Dootika Vats , Víctor Elvira

The presence of artificial intelligence (AI) in our society is increasing, which brings with it the need to understand the behavior of AI mechanisms, including machine learning predictive algorithms fed with tabular data, text or images,…

Machine Learning · Statistics 2025-06-06 Pedro Delicado , Cristian Pachón-García

Shapley values originated in cooperative game theory but are extensively used today as a model-agnostic explanation framework to explain predictions made by complex machine learning models in the industry and academia. There are several…

Machine Learning · Statistics 2024-04-15 Lars Henry Berge Olsen , Ingrid Kristine Glad , Martin Jullum , Kjersti Aas

Learning the joint dependence of discrete variables is a fundamental problem in machine learning, with many applications including prediction, clustering and dimensionality reduction. More recently, the framework of copula modeling has…

Machine Learning · Statistics 2013-11-15 Alfredo Kalaitzis , Ricardo Silva

The inferential model (IM) framework provides valid prior-free probabilistic inference by focusing on predicting unobserved auxiliary variables. But, efficient IM-based inference can be challenging when the auxiliary variable is of higher…

Statistics Theory · Mathematics 2015-01-20 Ryan Martin , Chuanhai Liu

Researchers in explainable artificial intelligence have developed numerous methods for helping users understand the predictions of complex supervised learning models. By contrast, explaining the $\textit{uncertainty}$ of model outputs has…

Machine Learning · Statistics 2023-11-01 David S. Watson , Joshua O'Hara , Niek Tax , Richard Mudd , Ido Guy