Optimizing combinatorial structures is core to many real-world problems, such as those encountered in life sciences. For example, one of the crucial steps involved in antibody design is to find an arrangement of amino acids in a protein sequence that improves its binding with a pathogen. Combinatorial optimization of antibodies is difficult due to extremely large search spaces and non-linear objectives. Even for modest antibody design problems, where proteins have a sequence length of eleven, we are faced with searching over 2.05 x 10^14 structures. Applying traditional Reinforcement Learning algorithms such as Q-learning to combinatorial optimization results in poor performance. We propose Structured Q-learning (SQL), an extension of Q-learning that incorporates structural priors for combinatorial optimization. Using a molecular docking simulator, we demonstrate that SQL finds high binding energy sequences and performs favourably against baselines on eight challenging antibody design tasks, including designing antibodies for SARS-COV.
@article{arxiv.2209.04698,
title = {Structured Q-learning For Antibody Design},
author = {Alexander I. Cowen-Rivers and Philip John Gorinski and Aivar Sootla and Asif Khan and Liu Furui and Jun Wang and Jan Peters and Haitham Bou Ammar},
journal= {arXiv preprint arXiv:2209.04698},
year = {2022}
}