English

Many Needles in a Haystack: Active Hit Discovery for Perturbation Experiments

Machine Learning 2026-05-12 v1

Abstract

High-throughput gene perturbation experiments can test several genetic interventions in parallel, yet experimental budgets remain limited. A central goal is hit discovery: identifying as many perturbations as possible whose phenotypic effect exceeds a predefined threshold. Pure exploration strategies are statistically inefficient, wasting budget on low-value regions. Bayesian optimization methods offer a principled alternative but target a single global optimum, over-exploiting dominant modes while neglecting other high-value regions. We formalize hit discovery as a sequential experimental design problem and propose Probability-of-Hit, an acquisition function that directly targets threshold exceedance by ranking candidates according to their posterior probability of being a hit. We prove asymptotic optimality of this approach and demonstrate strong empirical performance on both synthetic benchmarks and real biological immunology datasets, including up to 6.4% improvement over baselines on the Schmidt IL-2 dataset.

Keywords

Cite

@article{arxiv.2605.10196,
  title  = {Many Needles in a Haystack: Active Hit Discovery for Perturbation Experiments},
  author = {Andrea Rubbi and Arpit Merchant and Samuel Ogden and Amir Akbarnejad and Pietro Liò and Sattar Vakili and Mo Lotfollahi},
  journal= {arXiv preprint arXiv:2605.10196},
  year   = {2026}
}

Comments

To be published in International Conference on Machine Learning (ICML) 2026