2023 · 162 citations · 30 references
ComputingComputer ScienceIndustrial EngineeringComputer VisionMachine LearningSensor FusionAnomaly DetectionDeep LearningFeature-level FusionImage Feature EngineeringImage FusionMultimodal InteractionContrastive LearningFeature AlignmentFeature FusionImage Feature FusionMultimodal Feature FusionMultisensor Feature FusionPoint Cloud Feature FusionSegmentation
2D-based Industrial Anomaly Detection has been widely discussed, however, multimodal industrial anomaly detection based on 3D point clouds and RGB images still has many untouched fields. Existing multimodal industrial anomaly detection methods directly concatenate the multimodal features, which leads to a strong disturbance between features and harms the detection performance. In this paper, we propose Multi-3D-Memory (M3DM), a novel multimodal anomaly detection method with hybrid fusion scheme: firstly, we design an unsupervised feature fusion with patch-wise contrastive learning to encourage the interaction of different modal features; secondly, we use a decision layer fusion with multiple memory banks to avoid loss of information and additional novelty classifiers to make the final decision. We further propose a point feature alignment operation to better align the point cloud and RGB features. Extensive experiments show that our multi-modal industrial anomaly detection model outperforms the state-of-the-art (SOTA) methods on both detection and segmentation precision on MVTec-3D AD dataset. Code at github.com/nomewang/M3DM.
30
ImageNet: A large-scale hierarchical image database
Jia Deng, Wei Dong, Richard Socher et al. · 2009 IEEE Conference on Computer Vision and Pattern Recognition · 2009 · 61.8K citations
Laurens van der Maaten, Geoffrey E. Hinton · Journal of Machine Learning Research · 2008 · 35.7K citations
Martin A. Fischler, Robert C. Bolles · Communications of the ACM · 1981 · 25.5K citations · Full text
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov et al. · arXiv (Cornell University) · 2020 · 21.6K citations · Full text
Estimating the Support of a High-Dimensional Distribution
Bernhard Schölkopf, John Platt, John Shawe‐Taylor et al. · Neural Computation · 2001 · 5.9K citations