Detecting and counting harvested fish and identifying fish types in electronic monitoring system videos using deep convolutional neural networks

Abstract

Abstract The statistics of harvested fish are key indicators for marine resource management and sustainability. Electronic monitoring systems (EMSs) are used to record the fishing practices of vessels in recent years. The statistics of the harvested fish in the EMS videos are manually read and recorded later by operators in data centres. However, this manual recording is time consuming and labour intensive. This study proposed an automatic approach for prescreening harvested fish in the EMS videos using convolutional neural networks (CNNs). In this study, harvested fish in the frames of the EMS videos were detected and segmented from the background at the pixel level using mask regional-based CNN (mask R-CNN). The number of the fish was determined using time thresholding and distance thresholding methods. Subsequently, the types and body lengths of the fish were determined using the confidence scores and the masks predicted by the mask R-CNN model, respectively. The trained mask R-CNN model attained a recall of 97.58% and a mean average precision of 93.51% in terms of fish detection. The proposed method for fish counting attained a recall of 93.84% and a precision of 77.31%. An overall accuracy of 98.06% was obtained for fish type identification.