An End-to-End Steel Surface Defect Detection Approach via Fusing Multiple Hierarchical Features

TLDR

Defect detection requires simultaneous classification and precise localization, a composite task that is difficult to balance accurately and depends on costly, manually annotated datasets. The authors present a deep‑learning system for steel‑plate defect inspection. The system extracts multi‑stage CNN features, fuses them with a multilevel feature fusion network, then applies an RPN and ROI detector, and is trained on the newly introduced NEU‑DET dataset. On NEU‑DET, the method attains 74.8/82.3 mAP with ResNet34/50 using 300 proposals, and with only 50 proposals achieves 20 ft/s speed while retaining 92 % of that performance, demonstrating real‑time feasibility.

Abstract

A complete defect detection task aims to achieve the specific class and precise location of each defect in an image, which makes it still challenging for applying this task in practice. The defect detection is a composite task of classification and location, leading to related methods is often hard to take into account the accuracy of both. The implementation of defect detection depends on a special detection data set that contains expensive manual annotations. In this paper, we proposed a novel defect detection system based on deep learning and focused on a practical industrial application: steel plate defect inspection. In order to achieve strong classification ability, this system employs a baseline convolution neural network (CNN) to generate feature maps at each stage, and then the proposed multilevel feature fusion network (MFN) combines multiple hierarchical features into one feature, which can include more location details of defects. Based on these multilevel features, a region proposal network (RPN) is adopted to generate regions of interest (ROIs). For each ROI, a detector, consisting of a classifier and a bounding box regressor, produces the final detection results. Finally, we set up a defect detection data set NEU-DET for training and evaluating our method. On the NEU-DET, our method achieves 74.8/82.3 mAP with baseline networks ResNet34/50 by using 300 proposals. In addition, by using only 50 proposals, our method can detect at 20 ft/s on a single GPU and reach 92% of the above performance, hence the potential for real-time detection.