Integrated Clustering and Anomaly Detection (INCAD) for Streaming Data (Revised)
Abstract
Most current clustering based anomaly detection methods use scoring schema and thresholds to classify anomalies. These methods are often tailored to target specific data sets with "known" number of clusters. The paper provides a streaming clustering and anomaly detection algorithm that does not require strict arbitrary thresholds on the anomaly scores or knowledge of the number of clusters while performing probabilistic anomaly detection and clustering simultaneously. This ensures that the cluster formation is not impacted by the presence of anomalous data, thereby leading to more reliable definition of "normal vs abnormal" behavior. The motivations behind developing the INCAD model and the path that leads to the streaming model is discussed.
Cite
@article{arxiv.1911.00184,
title = {Integrated Clustering and Anomaly Detection (INCAD) for Streaming Data (Revised)},
author = {Sreelekha Guggilam and Syed M. A. Zaidi and Varun Chandola and Abani K. Patra},
journal= {arXiv preprint arXiv:1911.00184},
year = {2019}
}
Comments
13 pages; fixes typos in equations 5,6,9,10 on inference using Gibbs sampling