We propose a deterministic denoising algorithm for discrete-state diffusion models. The key idea is to derandomize the generative reverse Markov chain by introducing a variant of the herding algorithm, which induces deterministic state transitions driven by weakly chaotic dynamics. It serves as a direct replacement for the stochastic denoising process, without requiring retraining or continuous state embeddings. We demonstrate consistent improvements in both efficiency and sample quality on text and image generation tasks. In addition, the proposed algorithm yields improved solutions for diffusion-based combinatorial optimization. Thus, herding-based denoising is a simple yet promising approach for enhancing the generative process of discrete diffusion models. Furthermore, our results reveal that deterministic reverse processes, well established in continuous diffusion, can also be effective in discrete state spaces.