Stochastic Penalty-Barrier Methods for Constrained Machine Learning
Abstract
Constrained machine learning enables fairness-aware training, physics-informed neural networks, and integration of symbolic domain knowledge into statistical models. Despite its practical importance, no general method exists for the non-convex, non-smooth, stochastic setting that arises naturally in deep learning. We propose the Stochastic Penalty-Barrier Method (SPBM), which extends classical penalty and barrier methods to this setting via exponential dual averaging, a stabilized penalty schedule, and the Moreau envelope to handle non-smoothness. Experiments across multiple settings show that SPBM matches or outperforms existing constrained optimization baselines while incurring only linear runtime overhead compared to unconstrained Adam for up to 10,000 constraints.
Cite
@article{arxiv.2605.18618,
title = {Stochastic Penalty-Barrier Methods for Constrained Machine Learning},
author = {Adam Bosák and Andrii Kliachkin and Jana Lepšová and Gilles Bareilles and Jakub Mareček},
journal= {arXiv preprint arXiv:2605.18618},
year = {2026}
}