Fruit detection, tracking, and 3D reconstruction for crop mapping and yield estimation

Abstract

Through automated agricultural inspection, farmers can potentially achieve better productivity and accurately predict yields and crop quality. A variety of sensors can be used for agricultural inspection, but the cheapest and most information-rich is the video camera. We collect data in the field from a monocular camera fixed to a mobile inspection platform. For purposes of pineapple crop mapping and yield prediction, we propose an image processing framework for in-field fruit detection, tracking, and 3D reconstruction. We perform a series of experiments on feature point extraction using Harris, SIFT, and SURF features, feature point description using SIFT and SURF descriptors, feature point classification using SVMs, fruit region tracking using blob tracking, and 3D reconstruction using structure from motion and robust ellipsoid estimation techniques. We find that SURF feature points and descriptors provide the best tradeoff between processing time and classification accuracy and that the method is sufficiently accurate for fruit region detection. Our preliminary results for fruit region tracking and 3D fruit reconstruction are promising. We plan further work towards development of a useful aid to help farmers manage their farms.