Related papers: Listwise Deletion in High Dimensions

Models with dimension more than the available sample size are now commonly used in various applications. A sensible inference is possible using a lower-dimensional structure. In regression problems with a large number of predictors, the…

Given a dynamic network, where edges appear and disappear over time, we are interested in finding sets of edges that have similar temporal behavior and form a dense subgraph. Formally, we define the problem as the enumeration of the maximal…

Big-data applications often involve a vast number of observations and features, creating new challenges for variable selection and parameter estimation. This paper presents a novel technique called ``slow kill,'' which utilizes nonconvex…

We study loss functions that measure the accuracy of a prediction based on multiple data points simultaneously. To our knowledge, such loss functions have not been studied before in the area of property elicitation or in machine learning…

When facing data with imbalanced classes or groups, practitioners follow an intriguing strategy to achieve best results. They throw away examples until the classes or groups are balanced in size, and then perform empirical risk minimization…

We destroy a finite tree of size $n$ by cutting its edges one after the other and in uniform random order. Informally, the associated cut-tree describes the genealogy of the connected components created by this destruction process. We…

It is known that the scan statistic with variable window size favors the detection of signals with small spatial extent and there is a corresponding loss of power for signals with large spatial extent. Recent results have shown that this…

Hierarchical Model reduction and Proper Generalized Decomposition both exploit separation of variables to perform a model reduction. After setting the basics, we exemplify these techniques on some standard elliptic problems to highlight…

Selectivity estimation aims at estimating the number of database objects that satisfy a selection criterion. Answering this problem accurately and efficiently is essential to many applications, such as density estimation, outlier detection,…

We study the probability that a random polynomial with integer coefficients is reducible when factored over the rational numbers. Using computer-generated data, we investigate a number of different models, including both monic and non-monic…

In recent years, bootstrap methods have drawn attention for their ability to approximate the laws of "max statistics" in high-dimensional problems. A leading example of such a statistic is the coordinate-wise maximum of a sample average of…

We propose a high dimensional mean test framework for shrinking random variables, where the underlying random variables shrink to zero as the sample size increases. By pooling observations across overlapping subsets of dimensions, we…

The purpose of this paper is to study the fractal phenomena in large data sets and the associated questions of dimension reduction. We examine situations where the classical Principal Component Analysis is not effective in identifying the…

Let $(Y,X_1,...,X_m)$ be a random vector. It is desired to predict $Y$ based on $(X_1,...,X_m)$. Examples of prediction methods are regression, classification using logistic regression or separating hyperplanes, and so on. We consider the…

We consider a procedure to reduce simply generated trees by iteratively removing all leaves. In the context of this reduction, we study the number of vertices that are deleted after applying this procedure a fixed number of times by using…

Practitioners prune neural networks for efficiency gains and generalization improvements, but few scrutinize the factors determining the prunability of a neural network the maximum fraction of weights that pruning can remove without…

Testing the simplifying assumption in high-dimensional vine copulas is a difficult task. Tests must be based on estimated observations and check constraints on high-dimensional distributions. So far, corresponding tests have been limited to…

Taking projections of high-dimensional data is a common analytical and visualisation technique in statistics for working with high-dimensional problems. Sectioning, or slicing, through high dimensions is less common, but can be useful for…

Does the process of training a neural network to solve a task tend to use all of the available weights even when the task could be solved with fewer weights? To address this question we study the effects of pruning fully connected,…

High dimensional data analysis is known to be as a challenging problem. In this article, we give a theoretical analysis of high dimensional classification of Gaussian data which relies on a geometrical analysis of the error measure. It…