Auxiliary-Hyperparameter-Free Sampling: Entropy Equilibrium for Text Generation
Abstract
Token sampling strategies critically influence text generation quality in large language models (LLMs). However, existing methods introduce additional hyperparameters, requiring extensive tuning and complicating deployment. We present Entropy Equilibrium Sampling (EES), an auxiliary hyperparameter-free approach inspired by information theory that can dynamically adjust candidate sets by balancing normalized entropy with probability mass. We evaluate EES on both reasoning and generation tasks across a range of model architectures. Our results show that EES consistently performs well across temperature settings, delivering competitive accuracy and coherence while maintaining diversity. By eliminating the need for hyperparameter tuning, EES greatly simplifies deployment while improving performance. Code is available at https://github.com/shuanncai/EES
Keywords
Cite
@article{arxiv.2512.00789,
title = {Auxiliary-Hyperparameter-Free Sampling: Entropy Equilibrium for Text Generation},
author = {Xiaodong Cai and Hai Lin and Shaoxiong Zhan and Weiqi Luo and Hong-Gee Kim and Hongyan Hao and Yu Yang and Hai-Tao Zheng},
journal= {arXiv preprint arXiv:2512.00789},
year = {2025}
}