2021 · 18 citations · 15 references
EngineeringComputer ScienceComputer VisionDigital ImagingMachine VisionData CompressionLossy Data CompressionRate-distortion TheorySource CodingCompressionMotion CompensationMotion Field EstimationRate-distortion BoundsRate-distortion CodingRate-distortion FunctionRate-distortion OptimizationVideo Compression
Rate adaption is one of the decisive factors for the applications of video compression. Previous deep video compression methods are usually optimized for a single fixed rate-distortion (R-D) tradeoff. While they can achieve multiple bitrates by training multiple independent models, the achievable bitrates are limited to several discrete points on the R-D curve and the storage cost increases proportionally to the number of models. We propose a variable-rate scheme for deep video compression, which can achieve continuously variable rate by a single model, i.e., reaching any point on the R-D curve. In our scheme, two deep auto-encoders are used to compress the residual and the motion vector field respectively, which directly generate the final bitstream. The basic rate adaptation can be achieved by using the R-D tradeoff parameter to deeply modulate all the internal feature maps of the auto-encoders. In addition, other modules in our scheme, notably motion estimation and motion compensation, also affect the final bitrate indirectly. We further use the R-D tradeoff parameter to modulate them via a conditional map, thereby effectively improving the compression efficiency. We use a multi-rate-distortion loss function together with a step-by-step training strategy to optimize the entire scheme. The experimental results show the proposed scheme achieves continuously variable rate by a single model with almost the same compression efficiency as multiple fixed-rate models. The additional parameters and computation of our model are negligible when compared with a single fixed-rate model.
15
Multiscale structural similarity for image quality assessment
Zhou Wang, Eero P. Simoncelli, Alan C. Bovik · 2004 · 5.9K citations
FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks
Eddy Ilg, N. Michael Mayer, Tonmoy Saikia et al. · 2017 · 3.3K citations
Video Enhancement with Task-Oriented Flow
Jiajun Wu · International Journal of Computer Vision · 2019 · 1.3K citations · Full text