Uncertainty Makes It Stable: Curiosity-Driven Quantized Mixture-of-Experts
Abstract
Deploying deep neural networks on resource-constrained devices faces two critical challenges: maintaining accuracy under aggressive quantization while ensuring predictable inference latency. We present a curiosity-driven quantized Mixture-of-Experts framework that addresses both through Bayesian epistemic uncertainty-based routing across heterogeneous experts (BitNet ternary, 1-16 bit BitLinear, post-training quantization). Evaluated on audio classification benchmarks (ESC-50, Quinn, UrbanSound8K), our 4-bit quantization maintains 99.9 percent of full-precision F1 (0.858 vs 0.859) with 4x compression and 31 percent energy savings versus 8-bit, while both achieve statistical parity with full precision (p > 0.05). Crucially, curiosity-driven routing simultaneously improves accuracy and stability: on Quinn, F1 increases from 0.802 to 0.809 while cross-fold variance drops by 85 percent (p < 0.001, Levene's test), with reductions of 50 to 94 percent across datasets. The routing is self-organizing, with the high-precision 8-bit expert automatically receiving the most uncertain samples (20 percent lower confidence, p < 0.001), while lightweight experts handle easier inputs. Datasets with already low baseline variance show no artificial stability gain, confirming the mechanism targets genuine epistemic uncertainty rather than overfitting routing decisions. At 1.2M parameters, the framework provides interpretable, precision-aware routing suitable for safety-sensitive edge deployments where both accuracy and predictability are critical.
Cite
@article{arxiv.2511.11743,
title = {Uncertainty Makes It Stable: Curiosity-Driven Quantized Mixture-of-Experts},
author = {Sebastián Andrés Cajas Ordóñez and Luis Fernando Torres Torres and Mackenzie J. Meni and Carlos Andrés Duran Paredes and Eric Arazo and Cristian Bosch and Ricardo Simon Carbajo and Yuan Lai and Leo Anthony Celi},
journal= {arXiv preprint arXiv:2511.11743},
year = {2026}
}