2021 IEEE International Conference on Acoustics, Speech and Signal Processing
6-11 June 2021 • Toronto, Ontario, Canada
Extracting Knowledge from Information
| Paper ID | CI-3.2 | ||
| Paper Title | Frame-rate-aware Aggregation For Efficient Video Super-resolution | ||
| Authors | Takashi Isobe, Tsinghua University, China; Fang Zhu, New York University, United States; Shengjin Wang, Tsinghua University, China | ||
| Session | CI-3: Computational Photography | ||
| Location | Gather.Town | ||
| Session Time: | Thursday, 10 June, 15:30 - 16:15 | ||
| Presentation Time: | Thursday, 10 June, 15:30 - 16:15 | ||
| Presentation | Poster | ||
| Topic | Computational Imaging: [IMT] Computational Imaging Methods and Models | ||
| IEEE Xplore Open Preview | Click here to view in IEEE Xplore | ||
| Abstract | Video super-resolution, which aims at producing a high-resolution video from its corresponding low-resolution version, recently draws increasing attention. In contrast to the previous works that perform explicit motion estimation and compensation, we propose a novel deep neural network which performs implicit motion estimation with frame-rate-based temporal aggregation. Specifically, the input frames are first aggregated by a frame-rate-aware 3D convolution layer, where neighboring frames are integrated with the reference frame according to the corresponding frame rate. Then, the aggregated features are fed into several branches for further aggregation. Different branches correspond to a kind of motion rate, which provides complementary information to recover missing details in the reference frame. Extensive experiments demonstrate that our method is able to handle various motion types and achieves state-of-the-art performance on several benchmarks. In addition, our model is light-weight and requires an extremely less computational load than other state-of-the-art methods. | ||