English

Wrapper-Aware Rate-Distortion Optimization in Feature Coding for Machines

Image and Video Processing 2026-01-30 v1 Signal Processing

Abstract

Feature coding for machines (FCM) is a lossy compression paradigm for split-inference. The transmitter encodes the outputs of the first part of a neural network before sending them to the receiver for completing the inference. Practical FCM methods ``sandwich'' a traditional codec between pre- and post-processing neural networks, called wrappers, to make features easier to compress using video codecs. Since traditional codecs are non-differentiable, the wrappers are trained using a proxy codec, which is later replaced by a standard codec after training. These codecs perform rate-distortion optimization (RDO) based on the sum of squared errors (SSE). Because the RDO does not consider the post-processing wrapper, the inner codec can invest bits in preserving information that the post-processing later discards. In this paper, we modify the bit-allocation in the inner codec via a wrapper-aware weighted SSE metric. To make wrapper-aware RDO (WA-RDO) practical for FCM, we propose: 1) temporal reuse of weights across a group of pictures and 2) fixed, architecture- and task-dependent weights trained offline. Under MPEG test conditions, our methods implemented on HEVC match the VVC-based FCM state-of-the-art, effectively bridging a codec generation gap with minimal runtime overhead relative to SSE-RDO HEVC.

Keywords

Cite

@article{arxiv.2601.22070,
  title  = {Wrapper-Aware Rate-Distortion Optimization in Feature Coding for Machines},
  author = {Samuel Fernández-Menduiña and Hyomin Choi and Fabien Racapé and Eduardo Pavez and Antonio Ortega},
  journal= {arXiv preprint arXiv:2601.22070},
  year   = {2026}
}
R2 v1 2026-07-01T09:26:17.851Z