English
Related papers

Related papers: Estimating the Number of Clusters via Normalized C…

200 papers

On the basis of an analysis of previous research, we present a generalized approach for measuring the difference of plans with an exemplary application to machine scheduling. Our work is motivated by the need for such measures, which are…

Artificial Intelligence · Computer Science 2015-03-17 Martin Josef Geiger

In this paper, we define and study a new notion of stability for the $k$-means clustering scheme building upon the notion of quantization of a probability measure. We connect this notion of stability to a geometric feature of the underlying…

Statistics Theory · Mathematics 2018-03-09 Thibaut Le Gouic , Quentin Paris

When scholars suspect units are dependent on each other within clusters but independent of each other across clusters, they employ cluster-robust standard errors (CRSEs). Nevertheless, what to cluster over is sometimes unknown. For…

Methodology · Statistics 2025-11-12 Kentaro Fukumoto

Background: When planning a cluster randomized trial, evaluators often have access to an enumerated cohort representing the target population of clusters. Practicalities of conducting the trial, such as the need to oversample clusters with…

Methodology · Statistics 2024-09-19 Sarah E. Robertson , Jon A. Steingrimsson , Issa J. Dahabreh

We consider the problem of estimating the number of clusters (k) in a dataset. We propose a non-parametric approach to the problem that utilizes similarity graphs to construct a robust statistic that effectively captures similarity…

Methodology · Statistics 2025-06-13 Yichuan Bai , Lynna Chu

In this paper we use Clustering Method to understand whether stock market volatility can be predicted at all, and if so, when it can be predicted. The exercise has been performed for the Indian stock market on daily data for two years. For…

Computational Engineering, Finance, and Science · Computer Science 2016-04-19 Tamal Datta Chaudhuri , Indranil Ghosh

Existing approaches remain largely constrained by traditional distance metrics, limiting their effectiveness in handling random data. In this work, we introduce the first k-means variant in the literature that operates within a…

Most convex and nonconvex clustering algorithms come with one crucial parameter: the $k$ in $k$-means. To this day, there is not one generally accepted way to accurately determine this parameter. Popular methods are simple yet theoretically…

Machine Learning · Computer Science 2021-08-04 Sibylle Hess , Wouter Duivesteijn

In empirical work it is common to estimate parameters of models and report associated standard errors that account for "clustering" of units, where clusters are defined by factors such as geography. Clustering adjustments are typically…

Statistics Theory · Mathematics 2022-09-21 Alberto Abadie , Susan Athey , Guido Imbens , Jeffrey Wooldridge

We introduce a method for validation of results obtained by clustering analysis of data. The method is based on resampling the available data. A figure of merit that measures the stability of clustering solutions against resampling is…

Computational Physics · Physics 2007-05-23 Erel Levine , Eytan Domany

Measuring graph clustering quality remains an open problem. To address it, we introduce quality measures based on comparisons of intra- and inter-cluster densities, an accompanying statistical test of the significance of their differences…

Social and Information Networks · Computer Science 2020-03-20 Pierre Miasnikof , Alexander Y. Shestopaloff , Anthony J. Bonner , Yuri Lawryshyn , Panos M. Pardalos

Clustering approaches that utilize convex loss functions have recently attracted growing interest in the formation of compact data clusters. Although classical methods like k-means and its wide family of variants are still widely used, all…

A new interpoint distance-based measure is proposed to identify the optimal number of clusters present in a data set. Designed in nonparametric approach, it is independent of the distribution of given data. Interpoint distances between the…

Machine Learning · Computer Science 2022-10-18 Soumita Modak

In this paper, we test whether two datasets share a common clustering structure. As a leading example, we focus on comparing clustering structures in two independent random samples from two mixtures of multivariate normal distributions.…

Statistics Theory · Mathematics 2022-11-21 Chao Gao , Zongming Ma

A novel framework for consensus clustering is presented which has the ability to determine both the number of clusters and a final solution using multiple algorithms. A consensus similarity matrix is formed from an ensemble using multiple…

Machine Learning · Statistics 2014-08-06 Shaina Race , Carl Meyer

We study the problem of clustering sequences of unlabeled point sets taken from a common metric space. Such scenarios arise naturally in applications where a system or process is observed in distinct time intervals, such as biological…

Data Structures and Algorithms · Computer Science 2017-10-17 Tamal K. Dey , Alfred Rossi , Anastasios Sidiropoulos

Convex clustering, a convex relaxation of k-means clustering and hierarchical clustering, has drawn recent attentions since it nicely addresses the instability issue of traditional nonconvex clustering methods. Although its computational…

Methodology · Statistics 2019-01-01 Binhuan Wang , Yilong Zhang , Will Wei Sun , Yixin Fang

Mode clustering is a nonparametric method for clustering that defines clusters using the basins of attraction of a density estimator's modes. We provide several enhancements to mode clustering: (i) a soft variant of cluster assignment, (ii)…

Methodology · Statistics 2015-12-23 Yen-Chi Chen , Christopher R. Genovese , Larry Wasserman

Similarity is a fundamental measure in network analyses and machine learning algorithms, with wide applications ranging from personalized recommendation to socio-economic dynamics. We argue that an effective similarity measurement should…

Physics and Society · Physics 2015-12-07 Jian-Guo Liu , Lei Hou , Xue Pan , Qiang Guo , Tao Zhou

This paper proposes a novel method for determining the number of factors in linear factor models under stability considerations. An instability measure is proposed based on the principal angle between the estimated loading spaces obtained…

Methodology · Statistics 2024-09-13 Sze Ming Lee , Yunxiao Chen