Autoencoder-based Anomaly Detection in Streaming Data with Incremental Learning and Concept Drift Adaptation
Abstract
In our digital universe nowadays, enormous amount of data are produced in a streaming manner in a variety of application areas. These data are often unlabelled. In this case, identifying infrequent events, such as anomalies, poses a great challenge. This problem becomes even more difficult in non-stationary environments, which can cause deterioration of the predictive performance of a model. To address the above challenges, the paper proposes an autoencoder-based incremental learning method with drift detection (strAEm++DD). Our proposed method strAEm++DD leverages on the advantages of both incremental learning and drift detection. We conduct an experimental study using real-world and synthetic datasets with severe or extreme class imbalance, and provide an empirical analysis of strAEm++DD. We further conduct a comparative study, showing that the proposed method significantly outperforms existing baseline and advanced methods.
Cite
@article{arxiv.2305.08977,
title = {Autoencoder-based Anomaly Detection in Streaming Data with Incremental Learning and Concept Drift Adaptation},
author = {Jin Li and Kleanthis Malialis and Marios M. Polycarpou},
journal= {arXiv preprint arXiv:2305.08977},
year = {2023}
}
Comments
anomaly detection, concept drift, incremental anomaly detection, concept drift, incremental learning, autoencoders, data streams, class imbalance, nonstationary environments