English

Scalable Sampling for High Utility Patterns

Databases 2024-10-31 v1 Machine Learning

Abstract

Discovering valuable insights from data through meaningful associations is a crucial task. However, it becomes challenging when trying to identify representative patterns in quantitative databases, especially with large datasets, as enumeration-based strategies struggle due to the vast search space involved. To tackle this challenge, output space sampling methods have emerged as a promising solution thanks to its ability to discover valuable patterns with reduced computational overhead. However, existing sampling methods often encounter limitations when dealing with large quantitative database, resulting in scalability-related challenges. In this work, we propose a novel high utility pattern sampling algorithm and its on-disk version both designed for large quantitative databases based on two original theorems. Our approach ensures both the interactivity required for user-centered methods and strong statistical guarantees through random sampling. Thanks to our method, users can instantly discover relevant and representative utility pattern, facilitating efficient exploration of the database within seconds. To demonstrate the interest of our approach, we present a compelling use case involving archaeological knowledge graph sub-profiles discovery. Experiments on semantic and none-semantic quantitative databases show that our approach outperforms the state-of-the art methods.

Keywords

Cite

@article{arxiv.2410.22964,
  title  = {Scalable Sampling for High Utility Patterns},
  author = {Lamine Diop and Marc Plantevit},
  journal= {arXiv preprint arXiv:2410.22964},
  year   = {2024}
}

Comments

Accepted at 2024 IEEE International Conference on Big Data