Recently, deep learning (DL) based image transmission at the physical layer (PL) has become a rising trend due to its ability to significantly outperform conventional separation-based digital transmissions. However, implementing solutions at the PL requires a major shift in established standards, such as those in cellular communications. Application layer (AL) solutions present a more feasible and standards-compliant alternative. In this work, we propose a layered image transmission scheme at the AL that is robust to end-to-end (E2E) channel errors. The base layer transmits a coarse image, while the enhancement layer transmits the residual between the original and coarse images. By mapping the residual image into a latent representation that aligns with the structure of the E2E channel, our proposed solution demonstrates high robustness to E2E channel errors.