Transformer Networks for Continuous Gravitational-wave Searches

Wide-parameter-space searches for continuous gravitational waves using semi-coherent matched-filter methods require enormous computing power, which limits their achievable sensitivity. Here we explore an alternative search method based on training neural networks as classifiers on detector strain data with minimal pre-processing. Contrary to previous studies using convolutional neural networks (CNNs), we investigate the suitability of the transformer architecture, specifically the Vision Transformer (ViT). We establish sensitivity benchmarks using the matched-filter $\mathcal{F}$-statistic for ten targeted searches over a ten day timespan, and ten directed and six all-sky searches over a one day timespan. We train ViTs on each of these benchmark cases. The trained ViTs achieve essentially matched-filter sensitivity on the targeted benchmarks, and approach the $\mathcal{F}$-statistic detection probability of $p_{\mathrm{det}}$ = 90% on the directed ($p_{\mathrm{det}} \approx$ 85-89 %) and all-sky benchmarks ($p_{\mathrm{det}} \approx$ 78-88 %). Unlike the CNNs in our previous studies, which required extensive manual design and hyperparameter tuning, the ViT achieves better performance with a standard architecture and minimal tuning.