基于深度学习在移动 NPU 上实现功耗高效的视频超分辨率：Mobile AI & AIM 2022 挑战赛报告
图像与视频处理
2022-11-11 v1 计算机视觉与模式识别
摘要
视频超分辨率是移动设备上最受欢迎的任务之一，被广泛用于自动改善低码率和低分辨率视频流。尽管针对该问题已提出众多解决方案，但它们通常计算需求较高，在移动设备上表现出较低的帧率 (FPS) 和能效。在本次 Mobile AI 挑战赛中，我们着手解决这一问题，并提议参赛者设计一种针对低能耗优化的、用于移动 NPU 的端到端实时视频超分辨率解决方案。参赛者获得了包含用于 4 倍视频放大任务视频序列的 REDS 训练数据集。所有模型的运行时间和能效均在强大的 MediaTek Dimensity 9000 平台上进行了评估，该平台配备专用 AI 处理单元，能够加速浮点和量化神经网络。所有提出的解决方案均与上述 NPU 完全兼容，展现出高达 500 FPS 的帧率和 0.2 [瓦特 / 30 FPS] 的功耗。本文提供了挑战赛中开发的所有模型的详细描述。
引用
@article{arxiv.2211.05256,
title = {Power Efficient Video Super-Resolution on Mobile NPUs with Deep Learning, Mobile AI & AIM 2022 challenge: Report},
author = {Andrey Ignatov and Radu Timofte and Cheng-Ming Chiang and Hsien-Kai Kuo and Yu-Syuan Xu and Man-Yu Lee and Allen Lu and Chia-Ming Cheng and Chih-Cheng Chen and Jia-Ying Yong and Hong-Han Shuai and Wen-Huang Cheng and Zhuang Jia and Tianyu Xu and Yijian Zhang and Long Bao and Heng Sun and Diankai Zhang and Si Gao and Shaoli Liu and Biao Wu and Xiaofeng Zhang and Chengjian Zheng and Kaidi Lu and Ning Wang and Xiao Sun and HaoDong Wu and Xuncheng Liu and Weizhan Zhang and Caixia Yan and Haipeng Du and Qinghua Zheng and Qi Wang and Wangdu Chen and Ran Duan and Ran Duan and Mengdi Sun and Dan Zhu and Guannan Chen and Hojin Cho and Steve Kim and Shijie Yue and Chenghua Li and Zhengyang Zhuge and Wei Chen and Wenxu Wang and Yufeng Zhou and Xiaochen Cai and Hengxing Cai and Kele Xu and Li Liu and Zehua Cheng and Wenyi Lian and Wenjing Lian},
journal= {arXiv preprint arXiv:2211.05256},
year = {2022}
}
备注
arXiv admin note: text overlap with arXiv:2105.08826, arXiv:2105.07809, arXiv:2211.04470, arXiv:2211.03885