Automatic fish detection in underwater videos by a deep neural network-based hybrid motion learning system

TLDR

Underwater video fish detection must cope with poor lighting, varied fish orientation, seabed structures, background plant motion, and species‑dependent shape and texture diversity. The study proposes a unified approach to detect freely moving fish in unconstrained underwater environments using a Region‑Based Convolutional Neural Network. The method trains the network by fusing motion cues from background subtraction and optical flow with raw images to generate fish‑dependent candidate regions, and validates the approach on the Complex Scenes and LifeCLEF 2015 datasets. The hybrid approach achieves F‑Scores of 87.44 % on Complex Scenes and 80.02 % on LifeCLEF 2015, demonstrating its effectiveness for fish detection.

Abstract

Abstract It is interesting to develop effective fish sampling techniques using underwater videos and image processing to automatically estimate and consequently monitor the fish biomass and assemblage in water bodies. Such approaches should be robust against substantial variations in scenes due to poor luminosity, orientation of fish, seabed structures, movement of aquatic plants in the background and image diversity in the shape and texture among fish of different species. Keeping this challenge in mind, we propose a unified approach to detect freely moving fish in unconstrained underwater environments using a Region-Based Convolutional Neural Network, a state-of-the-art machine learning technique used to solve generic object detection and localization problems. To train the neural network, we employ a novel approach to utilize motion information of fish in videos via background subtraction and optical flow, and subsequently combine the outcomes with the raw image to generate fish-dependent candidate regions. We use two benchmark datasets extracted from a large Fish4Knowledge underwater video repository, Complex Scenes dataset and the LifeCLEF 2015 fish dataset to validate the effectiveness of our hybrid approach. We achieve a detection accuracy (F-Score) of 87.44% and 80.02% respectively on these datasets, which advocate the utilization of our approach for fish detection task.