IEEE Signal Processing Letters · 2020 · 166 citations · 26 references
ComputingComputer Science3D ModelingComputer VisionMachine Vision3D ReconstructionMotion EstimationVideo AnalysisConvolutionMotion CompensationSpatial Feature ExtractionSpatio-temporal RepresentationSpatiotemporal Feature ExtractionSuper-resolutionTemporal DimensionsVideo Super-resolution
The spatio-temporal information among video sequences is significant for video super-resolution (SR). However, the spatio-temporal information cannot be fully used by existing video SR methods since spatial feature extraction and temporal motion compensation are usually performed sequentially. In this paper, we propose a deformable 3D convolution network (D3Dnet) to incorporate spatio-temporal information from both spatial and temporal dimensions for video SR. Specifically, we introduce deformable 3D convolution (D3D) to integrate deformable convolution with 3D convolution, obtaining both superior spatio-temporal modeling capability and motion-aware modeling flexibility. Extensive experiments have demonstrated the effectiveness of D3D in exploiting spatio-temporal information. Comparative results show that our network achieves state-of-the-art SR performance. Code is available at: https://github.com/XinyiYing/D3Dnet.
26
Adam: A Method for Stochastic Optimization
Diederik P. Kingma, Jimmy Ba · UvA-DARE (University of Amsterdam) · 2014 · 84.8K citations · Full text
Learning Spatiotemporal Features with 3D Convolutional Networks
Du Tran, Lubomir Bourdev, Rob Fergus et al. · 2015 · 9.7K citations
Deformable Convolutional Networks
Jifeng Dai, Haozhi Qi, Yuwen Xiong et al. · 2017 · 6.9K citations
Deformable ConvNets V2: More Deformable, Better Results
Xizhou Zhu, Han Hu, Stephen Lin et al. · 2019 · 2.7K citations