English

jBOT: Semantic Jet Representation Clustering Emerges from Self-Distillation

Machine Learning 2026-04-27 v3 High Energy Physics - Experiment

Abstract

Self-supervised learning, in the context of foundation model training, is a powerful pre-training method for learning feature representations without labels, which often capture generic underlying semantics from the data and can later be fine-tuned for downstream tasks. In this work, we introduce jBOT, a pre-training method based on self-distillation for jet data from the CERN Large Hadron Collider, which combines local particle-level distillation with global jet-level distillation to learn jet representations that support downstream tasks such as anomaly detection and classification. We observe that pre-training on unlabeled jets leads to emergent semantic class clustering in the representation space. The clustering in the frozen embedding, when pre-trained on background jets only, enables anomaly detection via simple distance-based metrics, and the learned embedding can be fine-tuned for classification with improved performance compared to supervised models trained from scratch.

Keywords

Cite

@article{arxiv.2601.11719,
  title  = {jBOT: Semantic Jet Representation Clustering Emerges from Self-Distillation},
  author = {Ho Fung Tsoi and Dylan Rankin},
  journal= {arXiv preprint arXiv:2601.11719},
  year   = {2026}
}

Comments

Under review

R2 v1 2026-07-01T09:08:21.088Z