CircuitBuilder: From Polynomials to Circuits via Reinforcement Learning
Abstract
Motivated by auto-proof generation and Valiant's VP vs. VNP conjecture, we study the problem of discovering efficient arithmetic circuits to compute polynomials, using addition and multiplication gates. We formulate this problem as a single-player game, where an RL agent attempts to build the circuit within a fixed number of operations. We implement an AlphaZero-style training loop and compare two approaches: Proximal Policy Optimization with Monte Carlo Tree Search (PPO+MCTS) and Soft Actor-Critic (SAC). SAC achieves the highest success rates on two-variable targets, while PPO+MCTS scales to three variables and demonstrates steady improvement on harder instances. These results suggest that polynomial circuit synthesis is a compact, verifiable setting for studying self-improving search policies.
Cite
@article{arxiv.2603.17075,
title = {CircuitBuilder: From Polynomials to Circuits via Reinforcement Learning},
author = {Weikun K. Zhang and Rohan Pandey and Bhaumik Mehta and Kaijie Jin and Naomi Morato and Archit Ganapule and Michael Ruofan Zeng and Jarod Alper},
journal= {arXiv preprint arXiv:2603.17075},
year = {2026}
}
Comments
ICLR 2026 Workshop on AI with Recursive Self-Improvement