Spatiotemporal motion prediction in free-breathing liver scans via a recurrent multi-scale encoder decoder

In this work we propose a multi-scale recurrent encoder-decoder architecture to predict the breathing induced organ deformation in future frames. The model was trained end-to-end from input images to predict a sequence of motion labels. Targets were created by quantizing the displacement fields obtained from deformable image registration. We report results using MRI free-breathing acquisitions from 12 volunteers. Experiments were aimed at investigating the proposed multi-scale design and the effect of increasing the number of predicted frames on the overall accuracy of the model. The proposed model was able to predict vessel positions in the next temporal image with a mean accuracy of 2.07 (2.95) mm showing increased performance in comparison with state-of-the-art approaches.