Existing Large Language Models (LLMs) generate text through unidirectional autoregressive decoding methods to respond to various user queries. These methods tend to consider token selection in a simple sequential manner, making it easy to fall into suboptimal options when encountering uncertain tokens, referred to as chaotic points in our work. Many chaotic points exist in texts generated by LLMs, and they often significantly affect the quality of subsequently generated tokens, which can interfere with LLMs' generation. This paper proposes Self-Evaluation Decoding, SED, a decoding method for enhancing model generation. Analogous to the human decision-making process, SED integrates speculation and evaluation steps into the decoding process, allowing LLMs to make more careful decisions and thus optimize token selection at chaotic points. Experimental results across various tasks using different LLMs demonstrate SED's effectiveness.
@article{arxiv.2405.16552,
title = {SED: Self-Evaluation Decoding Enhances Large Language Models for Better Generation},
author = {Ziqin Luo and Haixia Han and Haokun Zhao and Guochao Jiang and Chengyu Du and Tingyun Li and Jiaqing Liang and Deqing Yang and Yanghua Xiao},
journal= {arXiv preprint arXiv:2405.16552},
year = {2024}
}
Comments
The relevant code will be released in subsequent versions