A Survey of Learn-to-Compute Paradigms for Rate-Distortion-Type Problems

Rate-distortion (RD) theory and its related formulations play a central role in understanding efficient information representation, but computing these quantities remains challenging in high-dimensional settings. Classical iterative methods such as the Blahut-Arimoto algorithm become impractical in high-dimensional domains due to the curse of dimensionality and the intractability of mutual-information terms. Recent advances in neural modeling and differentiable optimization offer a promising alternative through a learn-to-compute paradigm, in which probability distributions and objective functionals are represented by flexible neural parameterizations. This survey presents an overview of neural approaches for evaluating the RD-type objectives. We present three representative families of methods: variational inference, neural mutual-information estimation, and dual-form optimization. By reviewing their theoretical principles, algorithmic techniques, and consistency properties, we elucidate how these methods collectively transform classical RD-type problems into scalable differentiable objectives suitable for deep learning, though challenges remain in large-scale applications. Together, these perspectives offer promising avenues for scaling information-theoretic computation to complex, high-dimensional machine learning systems.