Multi-camera fusion and bird-eye view location mapping for deep learning-based cattle behavior monitoring

Abstract

Cattle behavioral monitoring is an integral component of the modern infrastructure of the livestock industry. Ensuring cattle well-being requires precise observation, typically using wearable devices or surveillance cameras. Integrating deep learning into these systems enhances the monitoring of cattle behavior. However, challenges remain, such as occlusions, pose variations, and limited camera viewpoints, which hinder accurate detection and location mapping of individual cattle. To address these challenges, this paper proposes a multi-viewpoint surveillance system for indoor cattle barns, using footage from four cameras and deep learning-based models including action detection and pose estimation for behavior monitoring. The system accurately detects hierarchical behaviors across camera viewpoints. These results are fed into a Bird's Eye View (BEV) algorithm, producing precise cattle position maps in the barn. Despite complexities like overlapping and non-overlapping camera regions, our system, implemented on a real farm, ensures accurate cattle detection and BEV-based projections in real-time. Detailed experiments validate the system's efficiency, offering an end-to-end methodology for accurate behavior detection and location mapping of individual cattle using multi-camera data.