English

Data clustering and noise undressing for correlation matrices

Statistical Mechanics 2009-11-07 v1 Disordered Systems and Neural Networks

Abstract

We discuss a new approach to data clustering. We find that maximum likelihood leads naturally to an Hamiltonian of Potts variables which depends on the correlation matrix and whose low temperature behavior describes the correlation structure of the data. For random, uncorrelated data sets no correlation structure emerges. On the other hand for data sets with a built-in cluster structure, the method is able to detect and recover efficiently that structure. Finally we apply the method to financial time series, where the low temperature behavior reveals a non trivial clustering.

Keywords

Cite

@article{arxiv.cond-mat/0101237,
  title  = {Data clustering and noise undressing for correlation matrices},
  author = {Lorenzo Giada and Matteo Marsili},
  journal= {arXiv preprint arXiv:cond-mat/0101237},
  year   = {2009}
}

Comments

8 pages, 5 figures, completely rewritten and enlarged version of cond-mat/0003241. Submitted to Phys. Rev. E