Learning agriculture keypoint descriptors with triplet loss for visual SLAM

Improving agriculture productivity by incorporating technology, machine learning, and robotics is a fundamental academic and industrial goal. One of the critical missing components toward this objective is the lack of accurate autonomous and precise robot navigation and localization methods. In this study, we develop and analyze new ideas to improve the performance of autonomous spatial localization and visual mapping (SLAM). In particular, we focus on the problem of keypoint detection and matching for visual (RGB) monocular imaging in challenging agricultural environments. These scenes are characterized by large numbers of repetitive patterns (e.g., foliage), extreme illumination conditions, and significant visual variations associated with seasonal landscape changes. We address these challenges by learning agricultural-specific keypoint descriptors in a self-supervised fashion. To that end, we implement a deep neural network model (DNN) and learn, in a data-driven fashion, image patch representations invariant to several spatial and visual transformations (e.g., changes in viewpoint and illumination). We collect data in natural agricultural areas and compare the proposed ideas with one of the state-of-the-art neural network-based solution, HardNet, and for two classical hand-crafted keypoint descriptors, ORB and SIFT, over three tasks, Patch Verification, Patch Retrieval, and Image Matching, improving in all tasks. Experiments show that our approach outperforms previous methods on agricultural environments, achieving a mean average precision of 90%, 96%, and 99%, respectively. This represents an improvement of 0.9%, 7.4%, and 0.1% compared with the neural network-based method.