English
Related papers

Related papers: Lurking Variable Detection via Dimensional Analysi…

200 papers

On the verge of the centenary of dimensional analysis (DA), we present a generalisation of the theory and a methodology for the discovery of empirical laws from observational data. It is well known that DA: a) reduces the number of free…

Classical Physics · Physics 2009-09-29 Michael Taylor , Angeles I. Diaz , Lucas A. Jodar-Sanchez , Rafael J. Villanueva-Mico

High dimensional statistical problems arise from diverse fields of scientific research and technological development. Variable selection plays a pivotal role in contemporary statistical learning and scientific discoveries. The traditional…

Statistics Theory · Mathematics 2009-10-08 Jianqing Fan , Jinchi Lv

For factor model, the involved covariance matrix often has no row sparse structure because the common factors may lead some variables to strongly associate with many others. Under the ultra-high dimensional paradigm, this feature causes…

Methodology · Statistics 2014-09-22 Junlong Zhao , Hongyu Zhao , Lixing Zhu

Bayesian change-point detection, together with latent variable models, allows to perform segmentation over high-dimensional time-series. We assume that change-points lie on a lower-dimensional manifold where we aim to infer subsets of…

Machine Learning · Statistics 2020-11-04 Lorena Romero-Medrano , Pablo Moreno-Muñoz , Antonio Artés-Rodríguez

Data exhibiting heavy-tails in one or more dimensions is often studied using the framework of regular variation. In a multivariate setting this requires identifying specific forms of dependence in the data; this means identifying that the…

Statistics Theory · Mathematics 2017-02-02 Bikramjit Das , Sidney I. Resnick

In this paper, we consider tests for ultrahigh-dimensional partially linear regression models. The presence of ultrahigh-dimensional nuisance covariates and unknown nuisance function makes the inference problem very challenging. We adopt…

Methodology · Statistics 2023-04-18 Hongwei Shi , Bowen Sun , Weichao Yang , Xu Guo

Unsupervised learning aims at the discovery of hidden structure that drives the observations in the real world. It is essential for success in modern machine learning. Latent variable models are versatile in unsupervised learning and have…

Machine Learning · Computer Science 2016-06-13 Furong Huang

This paper tackles the challenge of detecting unreliable behavior in regression algorithms, which may arise from intrinsic variability (e.g., aleatoric uncertainty) or modeling errors (e.g., model uncertainty). First, we formally introduce…

Machine Learning · Computer Science 2024-06-12 Andres Altieri , Marco Romanelli , Georg Pichler , Florence Alberge , Pablo Piantanida

A trend in all scientific disciplines, based on advances in technology, is the increasing availability of high dimensional data in which are buried important information. A current urgent challenge to statisticians is to develop effective…

Applications · Statistics 2010-09-30 Herman Chernoff , Shaw-Hwa Lo , Tian Zheng

The possibility that price dynamics is affected by its distance from a moving average has been recently introduced as new statistical tool. The purpose is to identify the tendency of the price dynamics to be attractive or repulsive with…

Physics and Society · Physics 2009-11-11 V. Alfi , F. Coccetti , M. Marotta , L. Pietronero , M. Takayasu

We develop a monitoring procedure to detect changes in a large approximate factor model. Letting $r$ be the number of common factors, we base our statistics on the fact that the $\left( r+1\right) $-th eigenvalue of the sample covariance…

Methodology · Statistics 2022-02-03 Matteo Barigozzi , Lorenzo Trapani

Inferring causal relationships from observed data is an important task, yet it becomes challenging when the data is subject to various external interferences. Most of these interferences are the additional effects of external factors on…

Machine Learning · Computer Science 2025-11-14 Ruichu Cai , Xiaokai Huang , Wei Chen , Zijian Li , Zhifeng Hao

This article develops a framework for testing general hypothesis in high-dimensional models where the number of variables may far exceed the number of observations. Existing literature has considered less than a handful of hypotheses, such…

Methodology · Statistics 2017-08-16 Yinchu Zhu , Jelena Bradic

Food authenticity studies are concerned with determining if food samples have been correctly labeled or not. Discriminant analysis methods are an integral part of the methodology for food authentication. Motivated by food authenticity…

Methodology · Statistics 2010-10-08 Thomas Brendan Murphy , Nema Dean , Adrian E. Raftery

The goal of Causal Discovery is to find automated search methods for learning causal structures from observational data. In some cases all variables of the interested causal mechanism are measured, and the task is to predict the effects one…

Machine Learning · Statistics 2024-01-11 Shuyan Wang

The paper considers variable selection in linear regression models where the number of covariates is possibly much larger than the number of observations. High dimensionality of the data brings in many complications, such as (possibly…

Methodology · Statistics 2016-11-29 Haeran Cho , Piotr Fryzlewicz

A fundamental task in science is to determine the underlying causal relations because it is the knowledge of this functional structure what leads to the correct interpretation of an effect given the apparent associations in the observed…

Artificial Intelligence · Computer Science 2024-08-02 Alexandre Trilla , Nenad Mijatovic

Emerging applications of sensor networks for detection sometimes suggest that classical problems ought be revisited under new assumptions. This is the case of binary hypothesis testing with independent - but not necessarily identically…

Information Theory · Computer Science 2019-03-27 Stefano Marano , Peter Willett

The problem of detecting anomalies in multiple processes is considered. We consider a composite hypothesis case, in which the measurements drawn when observing a process follow a common distribution with an unknown parameter (vector), whose…

Information Theory · Computer Science 2020-04-22 Bar Hemo , Tomer Gafni , Kobi Cohen , Qing Zhao

Dimensionless numbers and scaling laws provide elegant insights into the characteristic properties of physical systems. Classical dimensional analysis and similitude theory fail to identify a set of unique dimensionless numbers for a…

Fluid Dynamics · Physics 2022-12-28 Xiaoyu Xie , Wing Kam Liu , Zhengtao Gan