ACM Transactions on Graphics · 2023 · 11 citations · 28 references
ComputingComputer Science3D ModelingComputer VisionMachine LearningDeep LearningDocument Image AnalysisDocument LayoutGeometry ProcessingImage SegmentationShape DeformationDeformationDocument Layout AnalysisGeometric Distortion CorrectionMergingPredictionTransformer-based Segmentation
Single image rectification of document deformation is a challenging task. Although some recent deep learning-based methods have attempted to solve this problem, they cannot achieve satisfactory results when dealing with document images with complex deformations. In this article, we propose a new efficient framework for document flattening. Our main insight is that most layout primitives in a document have rectangular outline shapes, making unwarping local layout primitives essentially homogeneous with unwarping the entire document. The former task is clearly more straightforward to solve than the latter due to the more consistent texture and relatively smooth deformation. On this basis, we propose a layout-aware deep model working in a divide-and-conquer manner. First, we employ a transformer-based segmentation module to obtain the layout information of the input document. Then a new regression module is applied to predict the global and local UV maps. Finally, we design an effective merging algorithm to correct the global prediction with local details. Both quantitative and qualitative experimental results demonstrate that our framework achieves favorable performance against state-of-the-art methods. In addition, the current publicly available document flattening datasets have limited 3D paper shapes without layout annotation and also lack a general geometric correction metric. Therefore, we build a new large-scale synthetic dataset by utilizing a fully automatic rendering method to generate deformed documents with diverse shapes and exact layout segmentation labels. We also propose a new geometric correction metric based on our paired document UV maps. Code and dataset will be released at https://github.com/BunnySoCrazy/LA-DocFlatten .
28
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov et al. · arXiv (Cornell University) · 2020 · 21.6K citations · Full text
International conference on pattern recognition
Computers in Industry · 1983 · 444 citations · Full text
Learning to predict indoor illumination from a single image
Marc-André Gardner, Kalyan Sunkavalli, Ersin Yumer et al. · ACM Transactions on Graphics · 2017 · 332 citations
International Conference On Document Analysis and Recognition
1995 · 217 citations