2023 · 42 citations · 13 references
ComputingVisualizationComputer VisionMachine VisionVisual RepresentationDeep LearningImage CompressionRepresentation LearningScene ReconstructionVideo AnalysisVisual EncodingCompressionEncoder-decoder ArchitecturesImage EncodingInpaintingVideo CompressionVideo InpaintingVideo Representation Learning
We study how to represent a video with implicit neural representations (INRs). Classical INRs methods generally utilize MLPs to map input coordinates to output pixels. While some recent works have tried to directly reconstruct the whole image with CNNs. However, we argue that both the above pixel-wise and image-wise strategies are not favorable to video data. Instead, we propose a patch-wise solution, PS-NeRV, which represents videos as a function of patches and the corresponding patch coordinate. It naturally inherits the advantages of image-wise methods, and achieves excellent reconstruction performance with fast decoding speed. The whole method includes conventional modules, like positional embedding, MLPs and CNNs. We also introduce AdaIN to enhance intermediate features. Extensive experiments have demonstrated its effectiveness in several video-related tasks, such as video compression and video inpainting.
13
Adam: A Method for Stochastic Optimization
Diederik P. Kingma, Jimmy Ba · UvA-DARE (University of Amsterdam) · 2014 · 84.8K citations · Full text
DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · 2023 · 76.3K citations · Full text
Nonlinear Dimensionality Reduction by Locally Linear Embedding
Sam T. Roweis, Lawrence K. Saul · Science · 2000 · 15K citations